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Preface 


This  monograph  presents  a  framework,  methods,  and  tools  to  support 
capabilities  analysis  and  related  tradeoff  work  within  the  Department 
of  Defense  and  the  military  Services.  The  monograph  deals  with  choice 
and  risk. 

Some  parts  of  the  monograph  are  written  for  decisionmakers. 
These  sections  recommend  and  illustrate  principles  on  which  to  base 
terms  of  reference  for  major  capability  reviews  and  organization  of  sum¬ 
mary  presentations.  Other  parts  are  more  technical  and  are  intended 
for  readers  who  oversee  or  conduct  capability  evaluations  in  depth. 
The  monograph  in  its  entirety  should  also  be  of  interest  to  researchers 
interested  in  theory,  practice,  and  technology  for  supporting  high-level 
decisionmaking. 

The  research  reported  here  was  sponsored  primarily  by  Kenneth 
Krieg,  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (USD  (AT&L)),  and  by  James  Raleigh  Durham,  Director 
of  Joint  Advanced  Concepts  in  the  Office  of  the  Secretary  of  Defense 
(OSD  (AT&L)).  Most  of  the  study  was  completed  in  2006  and  early 
2007.  The  research  began  late  in  2005  in  response  to  an  earlier  request 
by  Michael  Wynne,  then  acting  USD  (AT&L),  for  a  generic  analytic 
framework  to  be  used  in  capability-area  reviews.  The  monograph  also 
reflects  earlier  research,  tool-development,  and  analysis  done  for  the 
Missile  Defense  Agency  (MDA)  under  the  sponsorship  of  David  Alt- 
wegg,  MDA’s  Deputy  for  Agency  Operations. 

Questions  and  comments  are  welcome  and  should  be  addressed 
to  the  senior  author,  Paul  K.  Davis.  He  can  be  reached  at  the  RAND 
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Corporation  in  Santa  Monica,  California  (telephone:  310-451-6912; 
email:  pdavis@rand.org). 

This  research  was  conducted  within  the  Acquisition  and  Technol¬ 
ogy  Policy  Center  of  the  RAND  National  Defense  Research  Institute, 
a  federally  funded  research  and  development  center  sponsored  by  the 
Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Com¬ 
batant  Commands,  the  Department  of  the  Navy,  the  Marine  Corps, 
the  defense  agencies,  and  the  defense  Intelligence  Community.  For 
more  information,  please  contact  the  Center  Director,  Philip  Anton,  at 
RAND  in  Santa  Monica  (telephone:  310-393-0411,  ext.  7798;  email: 
atpc-director@rand.org). 
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Summary 


Objectives 

The  research  reported  in  this  monograph  is  part  of  RAND’s  continu¬ 
ing  work  on  practical  theory  and  methods  for  capabilities-based  plan¬ 
ning  in  the  Department  of  Defense  (DoD)  and  other  organizations.  Its 
particular  contribution  is  to  describe  and  illustrate  in  some  detail  an 
analytic  framework  and  methodology  for  defensewide  capability-area 
reviews — including  DoD’s  experimental  Concept  Decision  Reviews 
and  related  evaluations  of  alternatives  (Krieg,  2007).  The  mono¬ 
graph  also  describes  newly  developed  enabling  tools — one  for  gener¬ 
ating  and  screening  preliminary  options  and  one  for  evaluating  in  a 
portfolio-analysis  structure  those  options  that  pass  screening.  Variants 
of  the  methods  can  be  applied  for  analysis  across  capability  areas  or  for 
strategic-level  defense  planning,  i.e.,  force  planning  to  establish  the 
overall  mix  and  balance  of  capabilities.  Finally,  the  monograph  illus¬ 
trates  concepts  with  applications  to  the  capability  areas  of  Global  Strike 
and  Ballistic  Missile  Defense  (BMD). 


A  Generic  Terms  of  Reference  for  Capability  Reviews 

Our  analytic  framework  is  summarized  by  Figure  S.l,  which  sketches  a 
generic  terms  of  reference  for  a  capability  review.  Although  straightfor¬ 
ward  and  seemingly  familiar  (just  another  process  diagram),  it  posits 
the  following  important  activities  that  are  defined  more  fully  below: 
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Figure  S.1 

Generic  Analytic  Process  for  Capability  Reviews 


Define  capability  area 


Evaluate  options  in  PMESII-sensitive 
political-military  portfolio  analysis 


^Effectiveness,  risk,  upside,  cost 


Characterize  shortfalls, 
opportunities,  "excesses" 


Add  more  improvement 
and  reduction  options 


Insights,  comparisons,  trades, 
and  possible  recommendations 
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1 .  Define  what  the  capability  area  “should  be”  to  best  support  the 
intentions  of  decisionmakers  and  to  address  problems  arising  at 
the  seams  between  capability  areas. 

2.  Construct  an  analytically  motivated  spanning  set  of  test  cases  to 
use  in  evaluating  the  options — not  for  optimum  effectiveness  in 
some  nominal  case,  but  for  flexibility,  adaptiveness,  and  robust¬ 
ness  (FARness)  across  the  range  of  possible  situations. 

3.  Define  concepts  of  operations  (CONOPS)  and  identify  criti¬ 
cal  components  of  those  CONOPS  to  use  in  evaluating  the 
options. 
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4.  Generate  diverse  options  and  screen  for  those  meriting  fuller 
analysis. 

5.  Identify  and  evaluate  options  in  a  portfolio-analysis  structure. 
With  our  apologies  for  using  the  current  alphabet  soup  of  ter¬ 
minology,  the  structure  should  include  political,  military,  eco¬ 
nomic,  social,  infrastructure,  and  information  (PMESII)  fac¬ 
tors.  The  options  should  draw  upon  diplomatic,  information, 
military,  and  economic  (DIME)  instruments  of  power. 

6.  Characterize  each  option  by  its  expected  contributions,  its  lim¬ 
itations  and  associated  risks,  and  its  upside  potential.  Where 
relevant,  point  out  where  the  option  arguably  creates  an  excess 
of  capability,  which  might  suggest  potential  bill-payers  in  sub¬ 
sequent  tradeoff  work. 

7.  Iterate  by  adding  or  modifying  options  and  analytic  structure 
based  on  initial  experience.  Continue  until  convergence  occurs 
or  time  runs  out. 

We  see  this  process  as  modernization  of  classic  activities  with 
which  good  analysts  are  familiar.  As  in  classic  analysis,  its  purpose  is 
to  inform  difficult  choices  in  the  presence  of  multiple  objectives  and 
economic  constraints.  We  believe  it  goes  well  beyond  classic  methods, 
however,  in  its  effective  confrontation  of  multiple  objectives,  uncer¬ 
tainty,  judgment,  risk,  and  PMESII  factors,  all  of  which  have  a  signifi¬ 
cant  impact  on  choice.  It  also  systematically  considers  a  broader  range 
of  options  than  is  the  norm. 


Establishing  a  Spanning  Set  of  Defense  Planning 
Scenarios 

The  next  step  of  the  process  involves  developing  a  spanning  set  of  test 
cases.  By  definition,  a  spanning  set  is  such  that  if  a  prospective  system 
tests  well  against  all  of  its  cases,  it  is  very  likely  that  the  system  will 
provide  appropriate  capability  in  any  of  the  diverse  situations  that  may 
arise  in  the  real  world.  That  is,  given  a  set  of  options  to  be  compared, 
the  spanning  set  stresses  the  various  options  in  all  of  the  key  dimen- 
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sions.  As  indicated  by  Figure  S.2,  we  find  an  approximate  spanning 
set  by  beginning  with  a  broadly  conceived  and  parameterized  scenario 
space.1  We  then  identify  analytically  distinguishable  regions  within  that 
space  and  define  a  particular  representative  case  for  each  region.  Such 
a  case  is  defined  by  a  name-level  scenario  and  the  values  of  key  param¬ 
eters  within  it  (e.g.,  “war  with  Country  X,  assuming  14  days  of  action¬ 
able  warning  before  war  begins  and  an  enemy  force  level  no  greater 
than  5  divisions”).  A  coarse-grained  spanning  set  would  need  relatively 
few  parameters;  a  higher-resolution  spanning  set  would  require  many 
more.  Developing  such  a  spanning  set  is  not  straightforward,  and  the 
quality  of  a  proposed  set  depends  on  the  quality  of  the  analytic  think¬ 
ing  that  underlies  it.2 

Given  a  spanning  set,  initial  analysis  should  then  inform  deci¬ 
sionmakers  about  the  relative  value,  feasibility,  and  cost  of  improving 

Figure  S.2 

Deciding  on  a  Spanning  Set  of  Cases 
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capabilities  for  the  different  regions.  Decisionmakers  can  then  make 
reasoned  judgments  about  what  capabilities  to  seek  actively.  These 
might  include  clearly  achievable  near-term  goals  and  some  optional 
“stretch  goals”  that  would  be  appropriate  for  later  in  an  evolutionary 
acquisition  but  that  are  not  technically  or  economically  implausible. 
Thus,  decisionmakers  should  carefully  express  “requirements”  (the 
initial-decisions  node  in  Figure  S.2)  in  light  of  the  initial  insights.  This 
results  in  a  smaller  spanning  set  of  cases,  one  that  spans  the  domain  in 
which  improvements  are  to  be  sought.  The  set  of  cases  can  be  embel¬ 
lished  to  become  defense  planning  scenarios  used  throughout  the 
Department  of  Defense  and  within  the  planning,  programming,  bud¬ 
geting,  and  execution  (PPBE)  process  in  particular.  In  today’s  DoD, 
decisions  on  planning  scenarios  are  made  by  the  Deputies  Advisory 
Working  Group  (DAWG).  Our  approach  seeks  to  provide  better  ana¬ 
lytical  support  for  those  decisions,  so  that  the  planning  scenarios  rep¬ 
resent  de  facto  requirements  rather  than  mere  expressions  of  diverse 
challenges. 


Tools  to  Support  Analysis 

Several  types  of  tools  are  needed  for  the  analysis  of  Figures  S.l  and  S.2. 
Figure  S.3  suggests  this  need  schematically.  Each  capability  area  needs 
analytic  support  from  a  broad  range  of  systems-engineering  models, 
diverse  models  and  simulations,  experiments,  and  other  forms  of  ana¬ 
lytical  information.  Together,  these  represent  foundational  knowledge. 
“Capability  models”  are  needed  at  a  somewhat  higher  level,  overlap¬ 
ping  with  aspects  of  systems  engineering.  These  are  typically  paramet¬ 
ric  models  suitable  for  systems  analysis  under  uncertainty.  They  may 
be  implemented  in  such  analyst-friendly  tools  as  Microsoft  Excel  and 
Analytica.  Another  class  of  tools  generates  options  and  then  screens 
for  the  candidates  that  merit  more-careful  assessment.  Finally,  there  is 
need  for  tools  to  assist  in  portfolio  analysis,  analysis  in  which  options 
are  evaluated  by  how — and  with  what  economic  efficiency — they  con¬ 
tribute  to  a  number  of  different  objectives  and  to  a  balancing  of  ambi¬ 
tions  and  risk-reduction.  As  discussed  in  the  main  text,  we  have  devel- 
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Figure  S.3 

Tools  Enabling  the  Portfolio-Analysis  Methodology 
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oped  and  applied  prototype  versions  of  a  special  option  generator  and 
screener  (Building  Blocks  to  Composite  Options  Tool  (BCOT))  and  a 
generic  portfolio-analysis  tool  (PAT). 

The  arrows  in  Figure  S.3  are  two-way  because  information  flows 
in  both  directions,  sometimes  bottom-up  and  sometimes  top-down. 
Moreover,  crucial  information  or  judgments  may  be  introduced  at  dif¬ 
ferent  levels  of  detail.  Detailed  analysis  is  sometimes  less  sound  and 
much  less  useful  than  higher-level  analysis;  at  other  times,  it  is  only 
with  detailed  analysis  and  precise  data  that  crucial  factors  can  be 
understood  adequately  and  effectiveness  assessed.  The  need  for  such 
a  multiresolution  family-of-tools  approach  should  be  seen  as  part  of  a 
best-practices  approach  to  capability  development.3 


Illustrative  Application:  A  Notional  Treatment 
of  Global  Strike 

Most  people  learn  from  examples,  working  through  particulars  first 
and  then  abstracting  to  more-general  principles.  In  this  monograph, 
we  describe  two  applications  of  our  methodology,  one  to  Global  Strike 
(or,  more  precisely,  to  what  DoD  refers  to  as  Global  Strike  Raid)  and 
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one  to  Ballistic  Missile  Defense  (BMD).  Some  readers  may  wish  to 
move  directly  from  Chapter  One  to  Chapters  Five  and  Six  for  these 
applications,  after  which  they  may  find  Chapters  Two  through  Four 
more  meaningful.  We  discuss  only  the  Global  Strike  example  in  this 
summary.  The  application  to  BMD,  however,  is  also  of  interest  because 
the  technical  challenges  are  so  difficult. 

Structuring  the  Global  Strike  Problem 

For  the  notional  Global  Strike  application,  we  addressed  what  DoD 
calls  Global  Strike  Raid:  small,  discrete,  conventional-weapon  strikes 
authorized  by  the  president  in  a  strategic  context.  We  concluded — after 
various  simplifications  (e.g.,  ignoring  cyberspace  options)  to  permit 
unclassified  analysis — that  a  reasonable  spanning  set  of  scenarios  and 
cases  would  comprise  fewer  and  more-challenging  versions  of  scenarios 
nominally  involving  attacks  on  mobile  targets,  such  as  mobile  inter¬ 
continental  ballistic  missiles  (ICBMs);  prompt  strikes  on  terrorist  lead¬ 
ers;  and  attacks  on  facilities  for  weapons  of  mass  destruction  (WMD). 

Our  approximate  spanning  set  has  a  distinct  analytical  basis.  It 
is  akin  to  what  a  design  engineer  would  identify  as  the  set  of  design 
points  and  performance  requirements  that  together  stress  all  the  key 
factors.  Figure  S.4  shows  a  simplified  example.  The  (horizontal)  x-axis 
measures  the  execution  time  required  for  a  given  system  to  perform 
the  Global  Strike  mission  after  an  order  to  do  so.  The  (vertical)  y-axis 
is  a  notional  composite  measure  of  the  target’s  character,  ranging  from 
a  simple  point  target  at  the  bottom  to  an  entire  complex  of  hard  and 
deeply  buried  targets  at  the  top.  Each  of  the  numbered  boundaries 
indicates  the  region  of  applicability  for  a  given  system.  Thus,  tacti¬ 
cal  aircraft  and  cruise  missiles  are  associated  with  the  region  to  the 
right  of  and  below  boundary  1  (black).  Similarly,  boundaries  2,  3,  and 
4  indicate  the  regions  applicable  to  bombers  (red),  ballistic  missiles 
(brown  dashed),  and  joint  Special  Operations  Forces  (SOF)  options 
(blue  dashed),  respectively.  The  solid  dots  indicate  representative  points 
to  which  test  scenarios  might  correspond.  That  is,  analysts  need  not 
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consider  every  point  within  a  given  region,  but  only  such  a  set  of  rep¬ 
resentative  points.* 

Figure  S.4  suggests  that  tactical  aircraft  and  cruise  missiles  (as 
well  as  bombers)  can  cover  much  of  the  space,  but  that  ballistic  mis¬ 
siles  are  uniquely  capable  among  kinetic  options  for  responses  in  one 
or  a  few  hours.  Bombers  have  special  capability  against  underground 
targets;  even  bombers,  however,  may  be  insufficient  against  the  more 
ambiguous  hard  and  deeply  buried  targets.  In  those  cases,  we  postu¬ 
lated  for  the  sake  of  this  example  that  there  might  be  special  weak 
points  that  might  be  discovered  by  classic  intelligence  work,  cyberspace 
methods,  or  SOF.  For  the  sake  of  the  diagram,  we  postulate  a  region 
requiring  some  unspecified  joint  operation  involving  a  combination  of, 
e.g.,  bombers,  missiles,  and  SOF. 

Figure  S.4 

Distinct  Regions  of  the  Global  Strike  Scenario  Space 


The  full  analysis  of  options  considers  the  times  required  for  alerting  and  maneuvering  of 
forces,  command  and  control,  targeting,  mission  planning,  and  decisionmaking.  Figure  S.4 
defines  analytical  regions,  showing  essentially  irreducible  distinctions  among  options. 
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Figure  S.4  provides  one  view  of  how  the  scenario  space  decom¬ 
poses,  but  there  are  other  dimensions.  After  considering  a  number 
of  those  dimensions,  we  concluded  that  our  spanning  set  needed  the 
three  classes  of  scenarios  mentioned  earlier,  associated  nominally  with 
attacking  (1)  mobile  targets,  (2)  terrorist  leaders,  and  (3)  WMD  facili¬ 
ties.  We  say  “nominally”  because  diverse  target  types  could  fall  into 
these  classes. 

As  Table  S.l  indicates,  the  major  factors  that  we  wanted  to  stress 
in  our  analysis  (left  column)  could  be  accommodated  by  particular 
cases  within  these  classes  of  scenario.  For  example,  we  used  scenario 
class  2  (attacking  terrorist  leaders  in  a  meeting  of  limited  duration)  to 
stress  timeliness  aspects  of  capability,  to  include  dependence  on  warn¬ 
ing.  We  addressed  all  of  the  factors  in  the  left  column  in  at  least  one 
scenario  class,  and  sometimes  in  more  than  one.  Some  of  these  factors 
are  more  political-strategic  in  nature  than  military-technical.  In  our 
example,  we  did  not  treat  several  important  dimensions  that  we  most 
certainly  would  have  treated  in  a  real  application.  These  were  (1)  size 
of  attack  needed,  (2)  target  mobility  at  the  time  the  strike  reaches  the 


Table  S.l 

Degree  to  Which  Scenario  Classes  Stress  Selected  Factors 


Factor 

Scenario 
Class  1 

(mobile  missiles) 

Scenario 

Class  2 

(terrorist  meeting) 

Scenario 

Class  3 

(WMD  facilities) 

Timeliness  and  dependence 

• 

•  ••• 

on  warning 

Bases  and  permissions 

•  • 

• 

•  •• 

Penetration  of  defenses 

•  ••• 

• 

•  •• 

Target  detection  and 

•  ••• 

• 

•  ••• 

characterization 

Target  destruction 

• 

• 

•  ••• 

Collateral  damage 

• 

•  • 

•  ••• 

Perceptions,  escalation  risk 

•  • 

•  • 

•  ••• 

NOTE:  The  numberof  bullets  indicates  the  degree  to  which  each  factor  is  stressed. 
The  number  of  bullets  shown  reflects  our  particular  analysis.  A  given  scenario  could 
be  chosen  to  be  more  stressful  in  other  ways.  For  example,  the  mobile-missile  and 
terrorist-meeting  scenarios  could  readily  stress  timeliness  and  collateral  damage, 
respectively. 
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target  area,  and  (3)  the  value  of  the  various  options  as  the  leading  edge 
of  a  larger  attack. 

Methodologically,  merely  specifying  a  single  concrete  scenario  of 
a  given  class  is  not  very  useful  because  of  stark  differences  in  chal¬ 
lenge,  depending  on  parameter  values  within  the  scenario.  How  many 
such  cases  need  to  be  considered  depends  on  the  problem  and  the 
level  of  resolution  needed,  which  in  turn  depends  on  the  type  of  deci¬ 
sion  being  made.  For  our  example,  we  concluded  that  much  could  be 
accomplished  with  six  cases,  as  described  below — relatively  favorable 
and  unfavorable  versions  of  the  three  scenario  classes. 

Outputs  of  Illustrative  Analysis 

The  outputs  of  this  analysis  are  illustrated  by  Figures  S.5,  which  is  based 
on  notional  and  somewhat  contrived  data.  The  figure  shows  a  top-level 
portfolio-analysis  summary  of  effectiveness.  It  assesses  options  (rows) 
by  each  of  seven  measures  (columns),  which  correspond  to  the  approxi¬ 
mate  spanning  set  of  three  scenarios  with  two  cases  each,  as  well  as 
a  measure  of  overall  risk  across  the  scenarios.  As  usual  in  such  score- 
cards,  red,  orange,  yellow,  light  green,  and  green  refer  to  very  poor, 
poor,  marginal,  good,  and  very  good,  respectively.  Letters  within  the 
cells  indicate  colors  for  the  benefit  of  those  using  gray-scale  prints  or 
for  color-blind  readers. 

Where  Do  the  Colors  Come  From?  Why  Should  We  Believe 
Them?  Although  effective  cognitively,  color  scorecards  can  be  off- 
putting  because  even  busy  decisionmakers  may  want  to  know  the  basis 
for  the  evaluations  and  may  want  to  have  a  reason  to  believe  that  the 
underlying  staff  work  was  rigorous.  As  discussed  in  Chapter  Five,  an 
important  feature  of  our  methodology  and  portfolio-analysis  tool  is 
that  it  allows  for  zooms  or  “drill-downs.”  Someone  viewing  Figure 
S.5,  for  example,  could  ask  about  the  basis  for  the  colors  in  the  first 
mobile-missile  case.  Successive  drill-downs  show  color  scorecards  dis¬ 
playing  underlying  factors,  calculations,  or  judgments.  If  even  more 
detail  is  needed,  it  is  best  presented  in  parametric  charts  characteristic 
of  classic  systems  analysis.  By  insisting  on  such  drill-down  capability, 
decisionmakers  can  create  a  powerful  incentive  for  staff-level  rigor.  An 
additional  reason  for  interactive  drill-down  is  that  portfolio  analysis 
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Figure  S.5 

A  Summary-Level  Portfolio-Analysis  Display  of  Effectiveness 


Options 


Measures  of  Option  Goodness:  Effectiveness  (Color) 
by  Scenario  Class  and  Overall  Risk 
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very  poor,  poor,  marginal,  good,  and  very  good. 
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should  be  conceived  not  as  a  way  to  show  cut-and-dried  results,  but 
rather  as  a  format  for  revealing,  discussing,  and  changing  assumptions  as 
the  result  of  conversations  with  decisionmakers.  Although  some  aspects 
of  portfolio  analysis  are  relatively  fact-based  and  “objective,”  many 
others  involve  strategic  judgments  about  relative  priorities  and  concerns, 
potential  constraints,  and  performance  levels  to  be  required — the  very 
items  for  which  decisionmakers  are  uniquely  responsible.  Thus,  analy¬ 
sis  should  include  serious  “straw-man”  assumptions  but  should  be  car¬ 
ried  forward  with  the  hope  for  and  expectation  of  substantive  interac¬ 
tion  and  iteration. 

Scorecards  Should  Tell  a  Correct  Story.  With  this  background 
in  the  philosophy  of  our  methodology,  let  us  proceed.  Although  the 
results  are  only  notional,  Figure  S.5  tells  a  story.  As  shown  in  the 
left  column,  the  options  considered  (a  subset  of  those  discussed  in 
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the  main  text)  include  procuring  certain  combinations  of  penetra¬ 
tion  aids  (penaids)  for  aircraft,  a  conventionally  armed  sea-launched 
ballistic  missile  (SLBM),  advanced  sensors  such  as  a  constellation  of 
space-based  synthetic-aperture  radars  (SARs)  and  automated-target- 
recognition  (ATR)  capability  for  aircraft,  an  advanced  bomber,  and  a 
hypothetical  advanced  insertion  vehicle  for  SOF. 

The  base  case  (reflecting  currently  programmed  systems)  asserts 
that  projected  ability  to  accomplish  the  various  missions  addressed  in 
the  three  scenarios  is  mostly  poor  for  the  year  2020.  A  realistic  assess¬ 
ment  might  differ,  but  the  methodology  is  well  served  by  a  story  with 
a  poor  baseline. 

Results  for  the  first  mobile-missile  column  improve  somewhat  (to 
orange)  with  a  combination  of  penetration  aids  and  advanced  sensors; 
they  improve  further  with  an  advanced  bomber  as  well  as  the  advanced 
sensors,  because  we  assumed  that  bombers  could  remain  in  the  target 
area  for  a  much  longer  period  of  time  (the  SOF  vehicle  is  actually 
irrelevant  to  effectiveness  in  this  scenario).  The  second  column  shows 
that  against  a  reactive  adversary  with  a  range  of  technical  and  tacti¬ 
cal  countermeasures,  results  would  not  be  improved  from  the  baseline. 
In  a  real  analysis,  of  course,  there  might  be  an  additional  and  more 
cheerful  column  for  results  assuming  U.S.  responses  to  the  adversary’s 
responses. 

In  the  terrorist  scenarios,  base-case  capability  is  projected  as  fairly 
good  (light  green)  for  terrorist  sites  generally  (third  column),  but  very 
poor  for  the  most  fleeting  targets  (fourth  column).  That  problem  could 
be  solved  with  a  suitable  ballistic  missile,  in  this  case  an  SLBM.  A  first 
version  of  such  a  capability  may  be  the  Conventional  Trident  Modifi¬ 
cation  (CTM),  which  is  proposed  in  the  president’s  budget  for  FY08. 

The  results  for  the  WMD-facilities  scenarios  indicate  that  penaids 
help  because  of  the  projected  quality  of  enemy  air  defenses,  but  in  our 
contrived  example,  they  are  not  sufficient  (effectiveness  is  shown  as 
yellow,  i.e.,  marginal).  If  we  zoomed  to  a  higher  level  of  detail  (as  we 
do  in  Chapter  Five),  we  would  see  that  the  reason  for  the  assessment  is 
that  some  WMD  facilities  are  likely  to  be  hard  and  deeply  buried,  so 
a  sizable  attack  causing  collateral  damage  might  be  required  if  accom¬ 
plished  with  aircraft  alone.  The  notional  evaluations  in  the  first  WMD 
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column,  however,  assume  that  intelligence  information  would  make  it 
possible  to  attack  those  successfully  with  a  hypothetical  joint  operation 
involving  a  combination  of  missiles,  bombers,  and  SOF — if  penetration 
of  defenses  by  SOF  were  enabled  by  procuring  the  specialized  insertion 
vehicle.  Such  an  attack  might  have  more  precision  and  might  be  able 
to  avoid  collateral  damage  caused  by  unintended  release  of  materials. 
Thus,  in  the  first  of  the  two  WMD-facilities  columns,  results  are  favor¬ 
able  for  the  options  that  provide  all  of  these  capabilities.  In  contrast, 
if  intelligence  were  inadequate  or  if  a  sizable  number  of  facilities  were 
essentially  invulnerable  to  a  modest  attack  of  any  sort  (second  WMD 
column),  results  would  remain  mixed  (yellow),  reflecting  the  presence 
of  good  and  bad  cases. 

Finally,  Figure  S.5  includes  a  column  for  a  kind  of  “overall”  risk 
associated  with  the  different  options.  This  summarizes  the  net  effect  of 
several  types  of  risk.  For  example,  the  other  columns  of  the  scorecard 
reflect  assessments  based  on  assumptions.  Flow  risky  are  the  assessments 
themselves?  Another  type  of  risk  arises  when  there  are  importantly  dif¬ 
ferent  subcases  within  a  scenario  case.  The  high-level  evaluation  is  an 
aggregation,  which  may  not  take  some  of  the  subcases  seriously  enough. 
The  risk  column,  then,  can  remind  us  of  fragile  assumptions.  By  zoom¬ 
ing  (or  drilling  down)  on  that  column,  we  would  see  (as  in  Chapter 
Five)  that  the  risks  vary  considerably  among  the  three  scenario  classes. 

Chapter  Five’s  discussion  of  this  material  also  covers  a  much 
larger  set  of  options.  This  is  important  because  in  some  cases  substan¬ 
tial  gains  could  be  achieved  less  expensively  with  operational  changes, 
such  as  forward  basing  of  bombers  or  different  choices  of  advanced  sen¬ 
sors  or  advanced  aircraft  (manned  or  unmanned).  Indeed,  one  part  of 
the  methodology  (the  fourth  step  in  Figure  S.l)  is  to  generate  potential 
options  that  may  not  have  been  suggested  by  proponents  of  the  various 
programs  and  to  screen  for  those  that  seem  potentially  attractive. 

Figure  S.6  presents  cost-effectiveness  charts  for  all  of  the  options 
that  we  considered  after  screening  out  the  less-attractive  ones.  Each 
point  represents  one  option,  and  there  are  thirteen  points  in  each  chart 
rather  than  just  six  for  the  subset  of  options  represented  in  Figure 
S.5.  The  colors  of  the  points  distinguish  options;  they  have  no  other 
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Figure  S.6 

Cost-Effectiveness  of  Scenarios  with  Different  Assumptions  and  Priorities 
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significance.  The  effectiveness  being  measured  is  some  kind  of  ag¬ 
gregation  over  the  effectiveness  of  the  seven  scorecard  columns  in 
Figure  S.5. 
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The  top-left  panel,  if  viewed  by  itself,  would  suggest  a  very  high 
cost-effectiveness  for  the  option  with  the  SLBM  and  penaids,  after 
which  we  see  diminishing  returns  (i.e.,  more-expensive  options  are  not 
substantially  more  effective).  The  real  point  of  Figure  S.6,  however,  is 
to  demonstrate  the  extreme  degree  to  which  results  depend  on  what  we 
call  “perspective.”  A  perspective  is  a  way  of  looking  at  the  choices.  Deci¬ 
sionmakers  may,  for  example,  have  long-term  rather  than  medium-term 
priorities;  they  may  be  more  concerned  about  fielding  useful  capability 
than  about  whether  there  would  be  eventual  countermeasures,  or  they 
might  be  very  reluctant  to  go  down  a  path  for  which  countermeasures 
can  already  be  identified.  They  might  be  much  more  concerned  about 
some  classes  of  future  scenario  than  others.  Some  decisionmakers  may 
be  interested  in  a  capability  only  if  its  expected  performance  is  quite 
high,  while  others  might  settle  for  the  earlier  deterrent  value  of  some¬ 
thing  with  poorer  performance. 

Mathematically,  we  represent  alternative  perspectives  with  a  com¬ 
bination  of  different  weighting  functions  and  performance  require¬ 
ments  used  in  the  computation  of  effectiveness.  The  top-left  panel 
treats  all  cases  as  equally  important.  The  panel  at  the  top  right  is  a  “best 
case”  for  the  options,  giving  weight  to  only  the  favorable  case  of  each 
scenario.  It  might  represent  the  perspective  of  a  decisionmaker  who 
believes — in  this  domain — that  achieving  capability  is  the  prime  con¬ 
sideration  and  that  the  United  States  would  be  able  to  develop  coun¬ 
ters  to  any  adversary  countermeasure.  From  this  perspective,  it  is  clear 
that  effectiveness  continues  to  rise  substantially  with  investment.  The 
bottom-left  panel  reflects  a  pessimistic  perspective  in  that  it  considers 
only  the  less-favorable  versions  of  the  scenarios,  the  versions  in  which 
the  adversary  employs  successful  countermeasures.  Finally,  the  panel 
at  the  bottom  right  shows  results  when  the  first  scenario  class  (attack¬ 
ing  mobile  missiles)  is  ignored  altogether — perhaps  because  decision¬ 
makers  conclude  that  it  is  an  implausible  scenario  for  a  very  limited 
global  strike.  In  this  case,  the  best  option  might  be  a  combination  of  an 
SLBM,  penaids,  and  SOF  vehicles,  which  attend  to  issues  in  scenario 
classes  2  and  3.  Further  expenditure  on  items  such  as  a  satellite  constel¬ 
lation  of  advanced  sensors  or  an  advanced  bomber  would  obviously  not 
be  cost-effective — at  least  for  the  purposes  of  Global  Strike  Raid. 
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The  relative  goodness  of  the  options  varies  a  great  deal  among  the 
four  perspectives,  illustrating  that  making  choices  is  highly  dependent 
upon  assumptions,  but  in  ways  that  can  be  summarized  compactly — a 
necessary  condition  for  the  kind  of  interactive  discussion  and  iteration 
mentioned  earlier. 

Much  of  the  value  of  our  methodology  resides  in  the  multiple 
levels  of  explanation  and  the  ability  to  see  crucial  assumptions  easily 
and  to  vary  them  systematically.  What  is  needed  here  is  not  standard 
sensitivity  analysis,  but  a  version  of  exploratory  analysis  generalized  to 
portfolio  work.  Exploratory  analysis  samples  across  the  entire  range  of 
assumption  combinations,  not  just  excursions  from  a  baseline  varying 
one  or  two  assumptions  at  a  time.  This  can  be  crucial  in  finding  the 
“corners”  of  the  assumption  space  where  capabilities  would  be  most 
stressed.  Adversaries  in  conflict  routinely  look  for  such  corners. 

Cross-Capability-Area  Analysis 

Our  analysis  demonstrates  the  limitations  of  a  managerial  approach 
that  attempts  to  do  resource-allocation  decisionmaking  one  capability 
area  at  a  time.  For  our  Global  Strike  example,  we  found  that  the  attrac¬ 
tiveness — from  a  parochial  Global  Strike  perspective — of  options  such 
as  the  advanced  manned  bomber  or  space-based  SARs  depends  sensi¬ 
tively  on  the  fraction  of  the  actual  acquisition  cost  that  is  charged  to 
the  Global  Strike  capability  area.  Knowing  what  that  fraction  should  be 
and  whether  those  systems  are  even  likely  to  be  acquired  at  all  requires 
separate  cross-cutting  analyses  in  which  the  attractiveness  of  the  big- 
ticket  acquisitions  is  assessed  simultaneously  for  each  of  the  capability 
areas  to  which  they  would  potentially  contribute.  If  the  overall  attrac¬ 
tiveness  is  deemed  high  by  the  Secretary  of  Defense,  then  the  issue 
becomes  one  of  allocating  costs.  Doing  so  may  be  straightforward  (e.g., 
charge  Global  Strike  for  only  the  marginal  cost  of  building  some  extra 
platforms  for  it)  or  somewhat  arbitrary — with  top  DoD  officials  decid¬ 
ing  on  whether  to  acquire  the  system,  how  capability  portfolios  should 
be  thought  of  as  sharing  its  costs,  and  allocating  funds  appropriately 
(in  some  cases  providing  “new  money”  to  one  or  more  capability  areas). 
None  of  these  decisions  can  be  made  within  a  single  capability  area. 
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Next  Steps  for  Research 

Looking  ahead,  we  recommend  further  research  in  four  areas  that  we 
believe  would  have  a  high  payoff  for  defense  planning. 

Analysis  Cutting  Across  Capability  Areas 

Extending  the  analysis  methodology  to  cut  across  capability  areas  is  cru¬ 
cial  for  the  reasons  mentioned  above.  DoD  leaders  intend  to  have  those 
responsible  for  capability  areas  do  “portfolio  balancing”  within  their 
respective  domains,  finding  bill-payers  as  well  as  additions.  However, 
as  noted  above,  in  many  cases,  it  is  necessary  to  do  cross-area  analysis 
to  properly  assess  the  value  of  a  new  system  or  activity,  the  implications 
of  cutting  back  on  another  system  or  activity,  or  the  way  to  think  about 
tradeoffs.  This  work  should  be  targeted  at  DoD’s  “tri-chair”  leaders  of 
capability  development:  the  USD  (AT&L),  the  Vice  Chairman  of  the 
Joint  Chiefs  of  Staff  (VCJS),  and  the  Director  of  Program  Analysis  and 
Evaluation  (DPA&E). 

Strategic-Level  Portfolio  Analysis 

Although  the  work  reported  here  was  performed  for  the  USD  (AT&L) 
and  his  staff,  with  others  in  mind  such  as  the  DPA&E  and  VCJS, 
the  same  methods  can  be  applied  at  the  strategic  level — both  within 
DoD  and  across  cabinet  departments.  Such  work  may  be  of  particu¬ 
lar  importance  in  the  near  future  as  the  activity  in  Iraq  winds  down 
and  policymakers  contemplate  alternative  relative  emphases  in  defense 
and  national  security  for  the  years  ahead — e.g.,  the  relative  empha¬ 
sis  on  the  “long  war,”  various  regional  instabilities,  and  the  long-term 
competition  in  East  Asia.  What  implications  should  there  be  for  rela¬ 
tive  investment  in  recapitalizing  the  armed  forces  after  some  years  of 
intense  activity  that  has  worn  them  down,  for  modernizing  forces  (even 
beyond  what  was  envisioned  by  late-1990s  transformation),  for  restruc¬ 
turing  for  low-intensity  conflicts  and  operations,  or  for  shifting  the 
pattern  of  military  environment-shaping  worldwide? 
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Advancing  the  Science  of  Exploratory  Analysis 

As  noted  throughout  this  monograph,  exploratory  analysis  is  essential 
for  pursuing  investment  strategies  that  exhibit  FARness,  i.e.,  strate¬ 
gies  that  are  flexible,  adaptive,  and  robust,  rather  than  optimized  for 
a  particular  image  of  the  future.  Such  exploration  goes  far  beyond  the 
limited  sensitivity  analysis  that  has  long  been  part  of  strategic  stud¬ 
ies.  Great  progress  has  been  made  over  the  past  decade  in  developing 
methods  for  exploratory  analysis  and  applying  them  to  actual  problems 
(Davis,  2002).  A  wave  of  new  challenges,  however,  is  posed  by  portfolio 
analysis.  In  portfolio  work,  the  uncertainties  have  a  different  character, 
both  substantively  and  technically.  Significant  research  will  be  needed 
to  extend  the  current  theory  and  methods  and  to  make  them  practi¬ 
cally  available  to  DoD.  Some  of  these  methods  can  call  upon  tools 
from  economic  theory  and  corporate  planning,  such  as  “real-option 
theory,”  but  many  are  more  nearly  unique  to  national-security  analysis. 
We  have  high  expectations  for  what  can  be  accomplished  here. 

Tool  Refinement 

In  the  course  of  the  work  described  in  this  monograph,  we  developed 
prototype  versions  of  RAND’s  Portfolio-Analysis  Tool  (PAT)  and  an 
option-generation-and-screening  tool  (BCOT).  We  then  demonstrated 
how  they  can  be  used  effectively,  even  in  their  current  form.  Further 
improvements,  however,  are  highly  desirable.  One  purpose  should  be 
to  make  them  easier  to  use,  understand,  modify,  and  maintain  by 
people  other  than  their  original  developers.  Another  purpose  should  be 
to  enrich  their  utility  for  exploratory  analysis  as  discussed  above  and  to 
incorporate  new  techniques  for  portfolio  analysis,  such  as  advances  in 
real-options  planning,  which  have  been  made  in  the  fields  of  econom¬ 
ics  and  business  planning. 
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Abbreviations  and  Acronyms 


ABP 

assumption-based  planning 

ACDT 

Advanced  Concept  Development  Test 

ALCM 

air-launched  cruise  missile 

AoA 

analysis  of  alternatives 

AT&L 

Acquisition,  Technology,  and  Logistics 

ATR 

automated  target  recognition 

BCOT 

Building  Blocks  to  Composite  Options  Tool 

BMD 

Ballistic  Missile  Defense 

BMDS 

ballistic-missile  defense  system 

C4ISR 

command,  control,  communications,  computer, 
intelligence,  surveillance,  reconnaissance 

CAMMD 

capabilities  analysis  model  for  missile  defense 

CAR 

Capability  Area  Review 

CAS 

complex  adaptive  systems 

CBP 

capabilities-based  planning 

COA 

course  of  action 

COCOM 

combatant  commander 

CONOPS 

concept  of  operations 

CTM 

Conventional  Trident  Modification 

CVBG 

combat  vehicle  battle  group 
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DAWG 

DCAR 

DDR&E 

DIME 

DoD 

DOFA 

DOTMLPF 

DPA&E 

DSB 

ECM 

EoA 

ESG 

FAR 

FARness 


FCB 

FCS 

FFRDC 

GPS 

HDBT 

IAMD 

ICBM 

IED 

ISR 

JCA 

JCAV 

JCIDS 


Deputies  Advisory  Working  Group 

Defense  Capability  Area  Review 

Director,  Defense  Research  and  Engineering 

diplomatic,  information,  military,  and  economic 

Department  of  Defense 

defense  of  friends  and  allies 

doctrine,  organization,  training,  materiel,  leadership 
and  education,  personnel,  and  facilities 
Director,  Program  Analysis  and  Evaluation 
Defense  Science  Board 
electronic  countermeasures 
evaluation  of  alternatives 
Engagement  Sequence  Group 
flexible,  adaptive,  and  robust 

an  attribute  of  strategy:  generating  flexible,  adaptive, 
and  robust  capabilities 

Functional  Capability  Board 

Future  Combat  System 

federally  funded  research  and  development  center 

Global  Positioning  System 

hard  and  deeply  buried  target 

Integrated  Air  and  Missile  Defense 

intercontinental  ballistic  missile 

improvised  explosive  device 

intelligence,  surveillance,  and  reconnaissance 

Joint  Capability  Area 

joint  common  aerospace  vehicle 

Joint  Capabilities  Integration  Development  System 
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JDAM 

JFACC 

JROC 

JSTPS 

JWCA 

LGB 

M&S 

MCP 

MDA 

MRM 

MTI 

NGLRS 

OSD 

OUSD 

PA&E 

PAT 

PGM 

PMESII 

PPBE 

S&T 

SAM 

SAR 

SecDef 

SIOP 

SIV 

SLBM 


joint  direct  attack  munition 
Joint  Forces  Air  Combat  Commander 
Joint  Requirements  Oversight  Council 
Joint  Strategic  Targeting  Planning  Staff 
Joint  Warfare  Capability  Area 
laser-guided  bomb 
modeling  and  simulation 
mission-capability  package 
Missile  Defense  Agency 
multiresolution  modeling 
moving-target  indicator 
next-generation  long-range  strike 
Office  of  the  Secretary  of  Defense 
Office  of  the  Under  Secretary  of  Defense 
Program  Analysis  and  Evaluation 
Portfolio-Analysis  Tool 
precision-guided  munition 

political,  military,  economic,  social,  infrastructure, 
and  information 

planning,  programming,  budgeting,  and  execution 

science  and  technology 

surface-to-air  missile 

synthetic-aperture  radar 

Secretary  of  Defense 

single  integrated  operations  plan 

special  insertion  vehicle  (a  hypothesized  advanced 
vehicle  for  Special  Operations  Forces) 
sea-launched  ballistic  missile 
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SLGSM 

sea-launched  global-strike  missile 

SOF 

Special  Operations  Forces 

SoS 

systems  of  systems 

SSBN 

nuclear-armed  ballistic-missile  submarine 

TCM 

Trident  Conventional  Missile 

UAV 

unmanned  aerial  vehicle 

USD  (AT&L) 

Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics 

VJCS 

WMD 

Vice  Chairman  of  the  Joint  Chiefs  of  Staff 
weapons  of  mass  destruction 

WSEG 

Weapons  Systems  Evaluation  Group 

Glossary  of  Terminology 


building-block  option.  An  element  of  military  capability  that  may 
be  acquired  (e.g.,  an  advanced  radar  or  an  advanced  bomber).  Most 
composite  options  involve  multiple  building  blocks. 

capabilities-based  planning  (CBP).  A  framework  for  planning  under 
uncertainty  that  provides  capabilities  suitable  for  a  wide  range  of 
modern-day  challenges  and  circumstances  within  an  economic 
framework  that  necessitates  choice. 

capability  area.  A  category  of  military  capabilities  that  can  usefully 
be  assessed  to  some  extent  independently  of  other  areas.  Examples 
include  Ballistic  Missile  Defense,  Strategic  Mobility,  and  Air-to-Air 
Combat. 

composite  option.  An  investment  option  that  involves  acquiring  one 
or  more  building-block  options. 

exploratory  analysis.  Analysis  that  systematically  examines  results 
as  a  function  of  different  plausible  values  of  key  input  parameters, 
varying  those  parameters  simultaneously  rather  than  individually  as 
is  done  in  normal  sensitivity  analysis.  In  exploratory  analysis,  where 
uncertainty  is  sometimes  ubiquitous  and  deep,  there  may  be  no 
meaningful  “best  estimate”  or  baseline,  other  than  for  comparison 
purposes. 

flexible,  adaptive,  and  robust  (FAR)  strategies.  Strategies  that  allow 
for  diverse  and  perhaps  unintended  missions  (flexibility);  effective 
operations  in  diverse  circumstances  (adaptiveness);  and  ability  to 
withstand  and  recover  from  adverse  events  or  shocks  (robustness). 
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perspective.  A  way  of  looking  at  the  problem  when  assessing  the 
“balance”  within  a  portfolio  of  investments.  A  perspective  is  specified 
mathematically  by  the  weighting  functions  and  performance 
requirements  used  in  the  calculation  of  overall  effectiveness  of 
options. 

portfolio.  A  collection,  as  in  investments  in  a  number  of  categories. 
A  strategic  portfolio  might  have  categories  for  different  functional 
classes  of  capability  or  categories  for  different  theaters  of  interest;  a 
portfolio  within  a  capability  area  would  have  categories  for  different 
contributors  to  that  area’s  overall  capability. 

portfolio  analysis.  Analysis  that  assesses  alternative  investment 
options  by  diverse  quantitative  or  qualitative  objectives,  including 
risk  mitigation.  The  analysis  aids  in  “balancing”  investments  within 
a  portfolio,  i.e.,  in  a  mix  of  instruments.  The  intent  is  to  address  all 
objectives  and  mitigate  all  risks,  but  to  different  degrees,  depending 
on  priorities,  budgets,  and  feasibilities. 

spanning  set.  A  small  set  of  scenarios  chosen  to  stress  a  design  or 
investment  plan  in  all  critical  dimensions.  An  option  that  does 
well  across  the  spanning  set  of  cases  should  do  well  in  real-world 
situations  (assuming  good  use  of  resources  at  the  time),  even  though 
those  situations  will  usually  differ  from  the  test  cases. 


CHAPTER  ONE 


Introduction 


Objectives 

This  monograph  describes  an  analytical  structure  and  related  methods 
for  conducting  defensewide  capability- area  reviews  for  the  Department 
of  Defense  (DoD).  Such  reviews  should  define  and  prioritize  needs, 
assess  alternative  ways  to  improve  capabilities,  illuminate  tradeoffs 
among  options  within  a  given  capability  area,  and  identify  decisions 
and  trades  cutting  across  capability  areas.  In  some  respects,  such  reviews 
are  comparable  to  what  were  once  called  mission-area  reviews.* 

Our  perspective  is  top-down  and  defensewide,  intended  to  be 
comfortable  to  the  Under  Secretary  of  Defense  for  Acquisition,  Tech¬ 
nology,  and  Logistics  (USD  (AT&L)),  the  Vice  Chairman  of  the  Joint 
Chiefs  of  Staff  (VJCS),  the  Director  of  Program  Analysis  and  Evalu¬ 
ation  (DPA&E),  and  other  decisionmakers  in  DoD  and  the  Services 
as  they  review  current  and  projected  capabilities  and  recommend  dif¬ 
ficult  choices  to  the  Secretary  of  Defense.  Such  choices  reflect  trade¬ 
offs  among  objectives  and  risks,  usually  within  the  reality  of  economic 
constraints. 

Decisionmaker  styles  vary  a  great  deal,  of  course,  as  do  the  con¬ 
texts  of  top-level  meetings,  including  decision  meetings  such  as  those 
for  go-or-no-go  decisions  on  capability  concepts.  Our  analytic  frame- 


Capability  areas  are  identified  at  different  times  for  different  purposes.  As  discussed  in 
Appendix  A,  the  Joint  Staff  has  a  comprehensive  taxonomy  of  Joint  Capability  Areas  (JCAs), 
but  these  are  not  always  convenient  for  programming  or  acquisition  work.  We  use  Global 
Strike  Raid  and  Ballistic  Missile  Defense  as  illustrative  capability  areas. 
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work  can  be  used  for  in-depth  analyses,  and  it  also  anticipates  the  need 
for  condensed,  meaty,  but  understandable  summary  discussions. 

Except  for  the  context-setting  discussion  in  Chapter  Two,  the 
monograph  is  not  about  process  issues  but  rather  focuses  on  analytic 
frameworks  and  methods.  The  monograph  is  one  element  in  RAND’s 
continuing  contributions  to  capabilities-based  planning  (CBP)  (Davis, 
Gompert,  and  Kugler,  1996;  Davis,  2002).  Most  of  the  research 
reported  here  was  completed  in  2006  and  early  2007. 


Structure  of  This  Monograph 

The  remainder  of  the  monograph  is  structured  as  follows.  Chapter 
Two  provides  background  on  DoD’s  capability-development  process, 
problems  with  that  process,  and  the  improvements  expected  from  the 
new  concept-development  process.  It  establishes  context  for  the  rest  of 
the  monograph,  although  the  methods  we  describe  could  be  used  in 
a  variety  of  capability-development  processes.  Chapter  Three  presents 
our  recommendations  for  a  generic  analytic  framework  and  process. 
Chapter  Four  discusses  tools  and  methods  needed  to  accomplish  the 
terms  of  reference.  Chapter  Five  walks  through  a  notional  application 
to  the  problem  of  Global  Strike.  Chapter  Six  sketches  a  similar  appli¬ 
cation  to  Ballistic  Missile  Defense.  Chapters  Five  and  Six,  then,  pro¬ 
vide  the  reader  with  a  rather  concrete  understanding  of  what  we  are 
suggesting  and  what  is  required.  Chapter  Seven  briefly  summarizes 
our  conclusions  and  suggests  desirable  next  steps  for  research.  Appen¬ 
dix  A  summarizes  the  Joint  Capability  Areas  used  by  the  Joint  Staff. 
Appendix  B  points  out  implications  of  this  work  for  systems  engineer¬ 
ing  and  modeling  and  simulation.  Appendix  C  describes  features  of 
RAND’s  Portfolio-Analysis  Tool  (PAT). 


CHAPTER  TWO 

Background:  The  Capabilities-Development 
Process 


Context:  DoD's  Capabilities-Development  Process 

In  recent  years,  the  Department  of  Defense  has  sought  to  create  a  pro¬ 
cess  suitable  for  the  capabilities-based  planning  (CBP)  mandated  in 
2001  and  reinforced  in  the  most  recent  Quadrennial  Defense  Reviews 
(Rumsfeld,  2001,  2006).  The  Aldridge  Report  (Joint  Defense  Capabil¬ 
ities  Study  Team,  2004)  reviewed  problems  with  the  preexisting  plan¬ 
ning  system  and  laid  out  objectives  and  schematics  for  an  improved 
process.  The  Joint  Staff  has  identified  both  operational  and  functional 
capability  areas  (see  Appendix  A)  and  has  organized  to  address  them 
systematically  and  comprehensively.  The  military  Services  have  re¬ 
organized  their  planning  systems  accordingly.  To  a  significant  degree, 
commonalities  of  vocabulary  and  categorizations  have  been  achieved. 
The  Joint  Staff  has  issued  a  draft  instruction  that  summarizes  the  over¬ 
all  current  process,  including  the  primary  activities,  their  relationships, 
and  the  inputs  and  outputs  for  the  various  activities  (Chairman  of  the 
Joint  Chiefs  of  Staff,  2007). 


Problems  with  the  Current  Process 

The  overall  process  is  described  in  the  Aldridge  Report.  Figure  2.1, 
taken  from  that  report,  shows  an  overview.  In  this  idealized  concept, 
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Figure  2.1 
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strategy  poses  challenges  to  planners,  who  develop  requirements  and 
capability  options.  Choices  must  then  be  made,  programs  and  bud¬ 
gets  developed,  and  capabilities  acquired.  Iteration  occurs  (see  bottom 
of  bgure)  as  the  result  of  time,  experience,  and  changes  in  the  envi¬ 
ronment.  The  authors  of  the  Aldridge  Report  acknowledged  that  the 
boxed  segment,  enhanced  planning,  was  where  miracles  must  happen. 
When  viewed  in  detail,  that  component  turns  out  to  be  complex,  bur¬ 
densome,  and  bureaucratic — a  problem  recognized  by  senior  officials 
and  officers.  DoD  leaders  have  been  trying  to  improve  that  process, 
and  such  efforts  are  continuing.  In  the  meantime,  success  depends  on 
high-level  good  will  and  collaborative  efforts  transcending  organiza¬ 
tional  structure. 
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A  sense  of  the  problems  is  given  by  Figure  2.2,  a  widely  used  offi¬ 
cial  depiction  in  2005.  Putting  aside  the  busy  nature  of  the  diagram,  a 
striking  feature  is  the  sequentiality  of  the  process,  with  separate  steps 
for  defining  operational  concepts,  conducting  functional-area  analy¬ 
ses,  conducting  functional-needs  analyses,  and  conducting  functional- 
solutions  analyses,  creating  an  initial  capability  document,  followed  by 
another  sequence  that  includes  analysis  of  refined  alternatives,  system 
engineering,  and  so  on.  This  sequentiality  is  represented  organization¬ 
ally  by  a  myriad  of  committees,  boards,  working  groups,  and  studies. 
Less  evident  but  even  more  troublesome  is  an  underlying  concept  of  the 
military  developing  “requirements”  and  potential  solutions  and  then 
passing  those  potential  solutions  over  the  transom  to  the  acquisition 
world  of  planning,  programming,  budgeting,  and  execution  (PPBE), 
where  decisions  must  be  made  on  resource  allocation.  The  separation 
was  described  rather  starkly  in  classroom  lectures  on  DoD’s  acquisition 
system  until  quite  recently.  Figure  2.3  was  used  in  an  internal  Penta¬ 
gon  course  on  the  JROC/JWCA*  processes  as  of  2001.  The  vertical  line 
separating  the  requirements  side  of  the  figure  from  the  acquisition  side 
was  referred  to  as  a  “firewall.” 

This  separation  of  supposedly  military  and  civilian  functions 
dates  back  to  an  unfortunate  consequence  of  the  Goldwater-Nichols 
Act  (U.S.  Congress,  1986).  Superficially,  it  may  seem  logical  for  the 
military  operators  to  establish  requirements  and  the  civilians  to  figure 
out  how  to  meet  those  requirements  and  pay  the  bill.  Upon  reflection, 
however,  that  concept  breaks  down.  In  the  real  world,  strategy,  require¬ 
ments,  technological  issues,  economics,  and  other  factors  need  to  be 
addressed  simultaneously — at  least  at  the  front  end  of  the  process. 
“Requirements”  are  not  handed  down  on  tablets  of  stone  but  should 
instead  be  seen  as  outputs  of  decisions  reached  after  consideration  of 
challenges,  desired  capabilities,  technical  feasibility,  economics,  orga¬ 
nizational  realities,  and  other  factors.4 

A  historical  problem  with  the  linear  construct  has  been  that 
requirement-setters,  when  working  in  isolation,  have  often  been  either 
too  conservative  (not  adequately  appreciating  future  needs,  technologi- 
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Figure  2.2 
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Figure  2.3 

Stark  View  of  Requirements  Versus  Acquisition 


SOURCE:  From  a  Joint  Staff  course  on  the  JROC/JWCA  process  in  2001 . 
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cal  opportunities,  and  the  likelihood  of  different  concepts  of  opera¬ 
tions)  or  naively  optimistic  (postulating  the  successful  rapid  develop¬ 
ment  of  capabilities  beyond  what  industry  could  realistically  gener¬ 
ate  on  the  timescale  imagined).5  Another  problem  has  been  that  the 
linear  construct  defers  serious  thinking  about  economic  considerations 
until — rather  late  in  the  process — the  favored  concept  is  finally  recog¬ 
nized  as  unaffordable.  In  such  cases,  valuable  time  has  been  lost,  time 
that  might  have  been  spent  on  developing  more  appropriate  concepts 
or  even  modifying  existing  capabilities  in  lieu  of  developing  something 
new. 


Historical  Successes  in  Capability  Development 

It  is  useful  in  thinking  about  the  way  ahead  for  DoD’s  capability- 
development  efforts  to  also  look  backward.  This  is  particularly  so 
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because  the  department  has  been  spectacularly  successful  over  the 
decades  in  many  of  its  developments.  What  factors  contributed  to  these 
successes? 

Table  2.1  lists  some  of  those  spectacular  developments.  An  over¬ 
arching  observation  is  that  almost  all  of  them  were  the  result  of  far¬ 
sighted  and  technologically  savvy  people  championing  developments 
outside  the  mainstream  bureaucratic  processes  of  the  time,  often 
achieving  their  successes  because  they  evaded  or  finessed  the  more  tidy, 
deliberate,  and  sequential  processes  of  their  time.6 

Another  observation  is  that  most  of  the  great  developments  included 
early  systems  analysis  and  systems  engineering  that  integrated  considerations 
of  strategy,  technological  promise  and  feasibility,  operational  effectiveness, 
risk,  and  economic  considerations.  There  was  also  a  good  deal  of  creative 
tension,  because  participants  had  different  perspectives  and  experi¬ 
ences.  Recognition  of  such  matters  underlies  part  of  DoD  leadership’s 
interest  in  the  Concept  Decision  Reviews  discussed  below,  which  focus 
on  the  “big- A”  issues  rather  than,  say,  those  of  how  best  to  purchase 
equipment  from  industry  (the  “small-a”  issues)  (Krieg,  2007). 

Although  numerous  examples  could  be  given,  we  shall  mention 
only  a  few: 

•  The  1967  STRAT-X  study  examined  a  broad  range  of  options  and 
greatly  influenced  development  of  U.S.  strategic  nuclear  forces 
over  subsequent  decades.7 

•  Top  Secret  studies  on  strategic  nuclear  command  and  control, 
conducted  by  the  Joint  Chiefs’  Weapons  Systems  Evaluation 
Group  (WSEG),  worked  through  the  complex  challenge  of  assur¬ 
ing  the  ability  to  command  and  control  the  nuclear  retaliatory 
forces  after  even  a  bolt-from-the-blue  attack  that  destroyed  many 
communication  mechanisms. 

•  The  multi-decade  development  of  the  Global  Positioning  System 
(GPS)  and  its  many  applications  to  navigation  and  precision 
fires  were  strongly  supported  by  studies  conducted  by  federally 
funded  research  and  development  centers  (FFRDCs)  and  indus¬ 
try,  including  very  early  studies  envisioning  with  remarkable 
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Table  2.1 

Some  Spectacularly  Successful  Capability  Developments 


Capability 

System 

Comment 

Sea-based  airpower  for 
fleet  defense,  strike, 
surveillance,  etc. 

Aircraft  carriers 

Developed  despite  lack  of 
interest  by  and  opposition  of 
the  "Battleship  Navy" 

Projection  of  ground 
forces  from  ships  into 
defended  land  areas 

Amphibious  operations 

Conceived  and  developed  by 
visionary  Marine  colonel 

Precision  fires 

Laser-guided  bomb  (LGB) 
and  joint  direct  attack 
munition  (JDAM) 

Resisted  consistently  by 

Service  PPBE  processes  and 
those  adhering  to  a  peculiarly 
strict  interpretation  of 
acquisition  regulations 

Maritime  operations  in 
littoral  areas 

Littoral  combat  ship 

Scorned  initially  but 
championed  by  Chief  of  Naval 
Operations 

Detection  and  tracking  of 
moving  ground  targets 

J-STARS 

Moved  directly  into  field  from 
R&D  during  first  Gulf  War 

Unmanned  surveillance 
and  targeting 

Global  Hawk  and 

Predator 

Resisted  by  Air  Force 

Tactical  mobility 

Stryker 

Championed  by  Army  Chief  of 
Staff  amid  controversy 

Strike,  penetration 

F-117,  B-2 

Championed  by  Air  Force 

Chief  of  Staff  and  DDR&E 

Precision  navigation, 

Global  Positioning  System 

Championed  by  DDR&E  and 

precision  fires 

(GPS) 

Secretary  of  Defense  and 
a  few  Air  Force  and  Navy 
leaders 

Early  warning  of  ballistic- 

DSP  (early  warning 

Resisted  by  most,  due 

missile  attack 

satellites) 

to  legacy  programs,  but 
supported  by  Air  Force  Chief 
of  Staff 

Submarine-based  nuclear 
retaliatory  forces 

SSBNs  (nuclear-armed 

ballistic-missile 

submarines) 

Imposed  on  Navy  by  president 

Assured  dissemination 

Strategic  nuclear 

Involved  largely  separate 

of  emergency  action 

command  and  control 

system,  thereby  avoiding 

messages,  even  in  surprise 
attack 

during  Cold  War 

many  sources  of  friction 

NOTE:  PPBE  =  planning,  programming,  budgeting,  and  execution;  DDR&E  =  Director, 
Defense  Research  and  Engineering.  In  the  period  referred  to,  DDR&E  had  power 
comparable  to  that  of  the  current  USD  (AT&L). 


prescience  many  of  the  radical  changes  that  would  occur  in  both 
military  and  civilian  operations.8 
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It  is  the  history  of  broadly  based,  cross-cutting,  front-end  analytic 
and  decision  efforts  that  we  primarily  wish  to  emphasize,  because  it 
bears  on  what  we  suggest  in  subsequent  chapters.  However,  it  is  also 
appropriate  to  list  some  of  the  other  characteristics  common  to  the 
great  developments. 

People.  The  great  developments  have  been  conceived,  envisioned, 
and  managed  by  remarkable  individuals.  The  consistent  emphasis  in 
interviews  with  “old-timers”  who  were  involved  in  those  developments 
is  on  people,  not  process.  This  is  in  contrast  to  the  prevailing  Pentagon 
culture,  which  emphasizes  consensus  and  process.  Indeed,  this  culture 
has  now  existed  for  so  long  that  the  very  intuition  of  many  people 
in  the  system  revolves  around  consensus  and  process,  with  resulting 
“requirements”  for  bloated,  exhausting  activities  and  no  one  having  the 
freedom  to  act. 

A  Sense  of  Mission.  Many  of  the  great  developments  were  strongly 
motivated  by  a  sense  of  mission,  such  as  defending  NATO  in  the  event 
of  a  massive  armored  invasion  by  the  Warsaw  Pact  or  deterring  major 
war  generally  through  a  combination  of  assured  nuclear  retaliation  and 
the  credible  ability  to  use  limited  nuclear  options  to  “reestablish  deter¬ 
rence”  if  conventional  war  began  in  Europe. 

The  Role  of  Vision.  The  great  developments  have  often  had  an 
accompanying  vision  that  provided  coherence  and  direction  over  time, 
a  vision  with  both  substance  and  legs.  These  coherent  visions  identified 
“thrusts,”  examples  of  which  we  shall  show  later.  These  were  developed 
and  honed  by  small  groups  of  top-notch  mid-level  “up-and-comers.”  A 
number  of  these  young  officers  later  became  well-known  flag  officers. 

Champions.  The  concepts  that  led  to  the  great  developments, 
however,  would  have  gone  nowhere  except  for  another  “people  issue,” 
the  championing  of  ideas  by,  e.g.,  the  Service  chief  and  senior  officials 
such  as  the  DDR&E,  who  in  earlier  years  had  power  comparable  to 
that  of  today’s  USD  (AT&L).9  Typically,  the  concepts  were  disruptive 
and  were  therefore  resisted  by  the  existing  organizations.  Leaders  had 
to  override  this  organizational  tendency  to  resist.  That  they  often  did  so 
is  perhaps  remarkable  to  those  familiar  with  the  “innovator’s  dilemma” 
in  industry  (Christensen,  1997),  but  defense  planning  has  objectives 
very  different  from  profit-making. 
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Strategic  Thinking.  Top  leaders  associated  with  the  great  develop¬ 
ments  thought  about  the  near,  mid-,  and  long  term  and  across  devel¬ 
opment  categories.  They  were  not  particularly  focused  on  science  and 
technology  (S&T);  they  did,  however,  value  it  greatly  and  pushed  inno¬ 
vation  in  diverse  ways  (e.g.,  Advanced  Concept  Development  Tests 
(ACDTs)  and  continuing  DARPA  activities).  They  thought  in  terms  of 
phased,  evolutionary  developments. 

Mainstream  Service  Opposition.  The  great  developments  (e.g., 
the  Global  Positioning  System,  Global  Hawk,  Predator,  the  Non- 
Lethal  Capability  Set  (NLCS),  F-16s)  usually  had  to  overcome  a  lack 
of  interest  or  even  strong  opposition  within  the  eventually  sponsoring 
Service.  Often,  Service  chiefs  played  a  critical  role  by  “reaching  down” 
and  championing  innovators  who  would  otherwise  have  lost  out  in 
the  competition  for  funding  and  priority.  Similarly,  Service  chiefs  were 
often  the  ones  who  faced  reality  about  the  need  for  tradeoffs.  Sometimes 
they  did  so  in  behind-closed-doors  cooperation  with  the  DDR&E  and 
the  Secretary  of  Defense  (SecDef),  without  consensus  within  the  Ser¬ 
vice.  Such  was  the  case  when  the  Air  Force  agreed  to  cooperate  in  a 
high-low-mix  approach  that  included  procurement  of  F-16s. 

SecDef  Intervention.  Upon  occasion,  the  SecDef  had  to  inter¬ 
vene,  overriding  the  preferences  of  the  Services  and  even  the  Service 
chiefs.  Sometimes  it  was  as  part  of  introducing  new  missions;  some¬ 
times  it  was  to  protect  “national”  programs  such  as  SSBNs;  sometimes 
it  was  to  reflect  conclusions  of  economic  analysis  (e.g.,  increasing  the 
rate  of  procurement  of  precision  weapons  or  unmanned  aerial  vehicles 
(UAVs)). 

Finessing  or  Evading  Processes.  Especially  relevant  is  the  obser¬ 
vation  that  the  great  developments  often  succeeded  despite,  rather  than 
because  of,  normal  processes.  Nothing  as  complex  as  the  Joint  Capa¬ 
bilities  Integration  Development  System  (JCIDS)  process  (see  Figure 
2.2)  existed  until  recent  years,  but  at  any  given  time,  the  then-normal 
process  was  often  regarded  as  too  burdensome  and  too  perilous.  The 
champions  of  the  new  ideas  found  ways  to  avoid  that  process  (e.g.,  by 
black  programs).  In  more-recent  times,  senior  officials  have  again  noted 
that  many  important  developments  are  dealt  with  outside  the  normal 
process  (e.g.,  the  current  task  force  on  improvised  explosive  devices 
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(IEDs)).  This  should  be  a  sobering  cautionary  for  those  who  seek  to 
solve  current  problems  by  perfecting  established  processes. 


Past  Recommendations  and  Recent  Changes 

Past  Recommendations  for  Change 

Against  this  background,  a  number  of  prominent  public  studies  have 
called  for  major  changes  in  the  DoD  acquisition  system.  These  include 
reports  of  the  Defense  Science  Board  (Defense  Science  Board,  2005, 
2007a,  2007b),  the  Kadish  Study  (Kadish,  2006),  and  a  Center  for 
Strategic  and  International  Studies  (CSIS)  report,  Beyond  Goldwater- 
Nichols  (Murdock,  Flournoy,  Campbell,  Chao,  Smith,  Witkowsky,  and 
Wormuth,  2005). 

In  2004-2005,  RAND  provided  a  highly  critical  independent 
review  of  DoD’s  capabilities-development  process,  characterizing  it  as 
bureaucratic  and  dysfunctional.10  It  recommended  major  changes,  par¬ 
ticularly  in  the  front  end: 

•  Integration  and  iteration ,  rather  than  lengthy  and  sequential  pro¬ 
cesses  (of  policy-setting,  assessment,  requirement-setting,  solution 
specification,  and  handover  to  acquisition). 

•  Having  a  single  group  approach  each  problem  area  early ,  a  group 
comprising  strategic  worriers,  operators,  technologists,  and  pro¬ 
gram  analysts,  although  perhaps  headed,  in  an  individual  case,  by 
an  ecumenical  “operator”  working  for  the  SecDef  and  the  Chair¬ 
man  of  the  Joint  Chiefs  of  Staff. 

•  More  extensive  early  use  of  systems  analysis  and  systems  engineering  to 
filter  by  feasibility,  define  threshold  capabilities  and  stretch  goals, 
provide  a  framework  for  assessment  and  tradeoffs,  and  provide 
information  with  which  to  monitor  and  adjust. 

An  overarching  recommendation  was  that  capability  assessments 
should  be  accomplished  on  a  DoD-wide  basis,  with  close  collaboration  of 
the  USD  (AT&L)  and  the  VJCS  . 
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This  was  in  contrast  to  redundant  and  sometimes  antagonistic 
Joint  Staff  and  OSD  processes.  We  suggested  the  name  Defense  Capa¬ 
bility  Area  Reviews  (DCARs)  and  anticipated  a  strong  role  for  AT&L, 
particularly  in  technology-informed  concept  development,  initial  sys¬ 
tems  engineering,  and  systems  analysis. 

In  its  2006  summer  study,  the  Defense  Science  Board  had  a 
number  of  recommendations  consistent  with  this  move  toward  more- 
integrated  planning.  They  included  recommendations  on  assuring 
early-in-the-process  influence  of  technological  expertise,  more-creative 
search  for  ways  in  which  to  exploit  technological  opportunities  arising 
in  the  civilian  sector,  and  the  creation  of  mission-oriented  portfolio 
managers  (Defense  Science  Board,  2007b). 

A  Recent  Change:  Concept  Decision  Reviews 

A  consensus  has  emerged  among  DoD  decisionmakers  on  many  of  the 
items  described  above,  including  the  need  for  integrated  work  in  which 
requirements,  acquisition,  and  resourcing  are  all  considered  at  the  same 
time  (Figure  2.3)  and  the  special  need  for  rejuvenated  efforts  of  this 
type  early  in  the  development  process.  It  is  hoped  that  early  top-level 
decisions  will  both  avoid  the  start  of  inappropriate  programs  and  build 
the  intellectual  and  organizational  consensus  needed  for  rapid  develop¬ 
ment  of  those  programs  that  are  approved. 

A  potentially  important  aspect  of  the  new  thinking  is  having  DoD 
potentially  move  toward  a  new  approach  currently  called  the  Concept 
Decision  Review,  illustrated  schematically  in  Figure  2.4  (Krieg,  2007). 
In  this  new  approach,  important  potential  capability  developments 
will  be  reviewed  and  the  concepts  will  be  accepted  or  rejected  by  a 
tri-chair  group  of  decisionmakers,  nominally  the  DepSecDef  or  USD 
(AT&L),  the  VJCS,  and  the  Director  of  OSD  (PA&E).  Their  deci¬ 
sion  will  be  based  on  substantial  analysis  prior  to  milestone  A  (a  new 
concept),  and  following  an  evaluation  of  alternatives  (EoA),  which  in 
turn  will  build  upon  products  of  the  JCIDS  process.  The  EoAs  that 
are  envisioned  would  be  significantly  different  from  the  analyses  of 
alternatives  (AoAs)  of  past  years.  They  would  be  broader  and  more 
creative  in  identifying  options  and  would  involve  the  integration  of 
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Figure  2.4 
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need,  technology,  concepts,  and  economics,  discussed  earlier.  Because 
decisions  would  also  occur  earlier,  they  would  likely  favor  planning  for 
sensible  evolutionary  development,  with  planned  times  for  subsequent 
decisions  at  branch  points. 

The  intent  is  then  to  have  intensive  technology  development  of 
an  approved  concept  prior  to  a  milestone  B,  which  could  be  reached 
more  rapidly  and  with  less  Office  of  the  Secretary  of  Defense  (OSD) 
management  oversight  than  has  typically  been  involved  in  going  from 
milestone  1  to  milestone  2  of  the  current  process.  As  indicated  at  the 
bottom  of  Figure  2.5,  the  intention  is  to  strongly  inform  early  concept 
development  by  technology  and  analysis. 
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Figure  2.5 
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CHAPTER  THREE 

A  Framework  and  a  Generic  Terms  of  Reference 


Given  the  larger  context  described  in  Chapter  Two,  we  next  develop 
an  analytic  framework  and  methodology  to  support  defensewide 
capability-area  reviews,  such  as  those  in  the  new,  experimental  Con¬ 
cept  Decision  Reviews.*  Some  readers  may  wish  to  move  forward  to 
Chapters  Five  and  Six  to  see  concrete  examples  before  reading  the 
more-generic  discussions  in  this  chapter. 


What  Should  a  Capability-Area  Review  Accomplish? 

The  capability-area  reviews  that  we  have  in  mind  are  specifically  for  top 
DoD  officials,  such  as  the  USD  (AT&L),  VJCS,  and  DPA&E.  It  is  not 
the  duty  of  such  officials  to  manage  programs.  Rather,  their  responsi¬ 
bilities  are  at  a  higher  level.  The  USD  (AT&L),  for  example,  shouldd 

•  Influence  initial  concept  decisions  to  approve  only  feasible  and 
economically  reasonable  concepts  with  good  threshold-  and 
stretch-capability  levels. 

•  Champion  bottom-up  innovation  from  the  Services. 

•  Strongly  influence  design  competitions,  tradeoffs,  and  choices. 


The  methods  could  be  used  within  other  processes,  such  as  the  Joint  Staff’s  JCIDS,  what 
OSD  (AT&L)  has  in  the  past  referred  to  as  capability-area  reviews  (CARs),  etc.  However,  the 
intent  is  to  have  a  defensewide  view. 

f  AT&I,  also  has  responsibilities  for  assuring  efficiency  of  acquisition  (what  is  sometimes 
referred  to  as  “small-a  acquisition”). 
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•  Assure  a  “systems  approach,”  both  in  the  design  of  individual  sys¬ 
tems  and  systems  of  systems  and  in  planning  for,  e.g.,  a  coherent 
logistics  component. 

•  Influence  priorities  among  systems  and  across  missions. 

•  Influence  science  and  technology  (S&T)  and  developmental 
experimentation. 

Perhaps  needless  to  say,  the  capability-area  reviews  should  also 
assure  that  jointness  is  taken  seriously  from  as  early  a  point  in  time  as 
possible:  Future  concepts  should  typically  be  “born  joint,”  in  the  sense 
of  being  defined  with  full  appreciation  of  joint  operations  and  the  value 
of  planning  for  cross-Service  cooperation,  coordination,  and  integra¬ 
tion  where  that  is  effective. 

With  this  in  mind,  we  describe  components  of  a  generic  terms  of 
reference  to  guide  a  capability  review.  We  developed  these  by  reviewing 
what  we  had  done  in  actual  application  projects,  by  generalizing,  and 
by  then  following  the  generic  terms  of  reference  for  a  new  application 
area  as  described  in  Chapter  Four. 

Figure  3.1  shows  the  key  elements  of  a  generic  terms  of  reference 
as  part  of  an  assessment  and  monitoring  process.  It  may  appear  at  first 
to  be  just  another  flow  diagram  of  the  sort  that  populates  so  many 
studies,  but  it  is  importantly  different  from  current  practice  in  several 
respects.  The  following  sections  discuss  each  of  these  in  turn. 


Elements  of  the  Generic  Analytic  Process 

Define  the  Capability  Area  and  Related  Missions 

What  Needs  to  Be  Done.  The  first  purpose  of  a  capability-area 
review  should  be  to  sharpen  the  definition  of  the  capability  area.  A  set 
of  official  joint  capability  areas  (JCAs)  has  been  developed  by  the  Joint 
Staff  (see  Appendix  A),  but  these  may  not  be  apt  for  a  specific  review. 
A  review  might  need  to  consider  an  area  that  is  more  limited  in  some 
respects  and  more  cross-cutting  in  others.  How  should  a  given  capabil¬ 
ity  area,  in  this  sense,  be  defined?  This  issue  may  be  nontrivial,  because 
of  pressures  to  narrow  and  simplify,  leaving  “seam  issues”  unaddressed 
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Figure  3.1 

A  Generic  Analytic  Process  for  Defense  Capability-Area  Reviews 
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and  the  mission’s  range  too  limited  (“We’d  never  do  that.”  “We’d  never 
be  in  that  situation.”  “That’s  so-and-so’s  job,  isn’t  it?”).  Defining  a 
cross-cutting  area  is  often  troublesome  because  it  implies  participation 
of  several  organizations. 

The  other  part  of  this  first  component  is  identifying  the  opera¬ 
tional  missions  associated  with  the  capability  areas  so  as  to  characterize 
what  capabilities  exist  and  what  are  needed. 

Semantic  issues  intrude.  The  word  “capability”  has  dual  meanings 
in  DoD  planning.  The  Joint  Staff’s  recent  draft  instruction  defines  it 
as  follows: 
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Capability  (1):  The  ability  to  achieve  a  desired  effect  under  speci¬ 
fied  standards  and  conditions  through  combinations  of  means 
and  ways  to  perform  a  set  of  tasks.  It  is  defined  by  an  operational 
user  and  expressed  in  broad  operational  terms  in  the  format  of  a 
joint  or  initial  capabilities  document  or  a  joint  doctrine,  organiza¬ 
tion,  training,  materiel,  leadership  and  education,  personnel,  and 
facilities  (DOTMLPF)  change  recommendation. 

This  definition  is  useful  in  one  context,  but  it  is  not  what  a  Secre¬ 
tary  of  Defense  typically  has  in  mind  when  he  uses  the  term.  Often,  he 
would  have  in  mind  something  like 

Capability  (2):  The  ability  to  achieve  a  type  of  desired  effect  (e.g., 
accuracy  or  speed  of  action),  perhaps  through  a  number  of  differ¬ 
ent  operational  mechanisms  over  time. 

Thus,  he  might  refer  to  stealth  or  precision  fires  as  “capabilities.” 
To  avoid  confusion  in  this  monograph,  we  use  “broad  capability”  to 
refer  to  the  second  meaning  above. 

The  relationship  between  these  usages  can  be  understood  by 
attempting  to  give  substance  to  the  second  definition.  Precisely  what 
would  someone  mean  when  talking  about  stealth  or  precision?  How 
much,  in  what  context,  and  as  measured  by  what?  Those  questions  are 
answered  by  considering  possible  operational  missions,  decomposing 
them  into  the  tasks  to  be  accomplished,  and  looking  for  natural  met¬ 
rics.11  The  risk  exists,  however,  that  the  missions  and  tasks  will  be  so 
narrowly  and  traditionally  defined  that  they  miss  the  point  of  the  new 
broad  capability.  The  difficulties  can  be  illustrated  by  some  historical 
examples. 

Past  Examples  of  Defining  and  Scoping.  The  case  of  strategic 
nuclear  offensive  forces  was  an  early  example  of  thinking  through  what 
constitutes  a  capability  area  and  what  the  range  of  contributors  to  it 
should  be.  The  choices  made  were  not  inevitable.  Early  on,  the  United 
States  might  have  given  the  entire  task  of  strategic  offense  to  the  Air 
Force,  perhaps  using  a  combination  of  bombers  and  ballistic  missiles. 
In  the  1960s  or  1970s,  it  might  have  decided  to  rely  entirely  on  sea- 
launched  ballistic  missiles  (SLBMs).  Instead,  the  nation  chose  to  go 
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with  a  triad  concept  of  independently  survivable  ground-based  inter¬ 
continental  ballistic  missiles  (ICBMs),  SLBMs,  and  manned  bomb¬ 
ers — later  expanded  to  include  air-launched  cruise  missiles  (ALCMs) 
and  SLBMs.12 

The  missions  and  priorities  chosen  for  the  forces  were  also  not 
inevitable.  From  the  era  of  President  Eisenhower  onward,  the  core  con¬ 
cepts  were  deterrence  through  assured  second-strike  capability  and 
then  extended  deterrence.  Counterforce  capability  was  always  impor¬ 
tant  as  well. 

From  early  on,  strategic  nuclear  offensive  forces  were  seen  from 
what  would  today  be  called  a  joint  perspective.  Conceived  in  the  late 
1950s  and  created  in  I960,  the  Joint  Strategic  Targeting  Planning  Staff 
(JSTPS)  generated  a  single  integrated  operations  plan  (SIOP)  in  which 
synergies  were  considered  and  increasingly  exploited.  For  example, 
early  strikes  by  SLBMs  on  major  Soviet  radars  would  make  subsequent 
bomber  penetration  much  easier. 

As  a  second  historical  example  of  the  importance  of  defining  a 
capability  area,  consider  the  missions  of  achieving  and  maintaining 
air  superiority  and  conducting  strike  operations.  As  the  result  of  an 
evolutionary  process,  today’s  military  has  a  Joint  Forces  Air  Combat 
Commander  (JFACC)  who  coordinates  all  air  operations — something 
very  different  from  the  situation  decades  ago.  The  change  did  not  come 
about  easily.  However,  the  “seam”  problems  have  been  addressed  and 
are  more  nearly  solved  than  they  were  in  past  years.13 

The  examples,  then,  illustrate  why  the  definitional  stage  of 
defensewide  capability  reviews  is  significant.  The  next  stage  is  develop¬ 
ing  an  analytic  framework. 

Characterize  Operational  Needs  in  a  Scenario  Space 

The  next  component  of  our  generic  terms  of  reference  (Figure  3.1)  is 
identifying  current  and  future  operational  needs  within  an  analytic 
framework  suitable  for  broad,  creative,  and  rigorous  thinking.  Such 
a  framework — fundamental  to  strong  versions  of  capabilities-based 
planning  (Davis,  2002) — involves  parameterizing  a  broad  scenario 
space.  However,  it  is  also  valuable  and  perhaps  even  essential  to  refer  to 
concrete,  specific  scenarios.  In  this  section,  we  discuss  how  the  broad, 
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parametric,  and  concrete-scenario  aspects  relate  to  each  other.  We  start 
with  the  broader  discussion  and  then  become  more  concrete. 

There  are  two  primary  challenges  in  conceiving  the  parametric 
scenario  space:  effectively  confronting  massive  uncertainty  and  con¬ 
sidering  all  dimensions  of  a  “scenario” — not  just  those  convenient  for 
quantitative  calculations. 

Planning  Under  Uncertainty.  The  Department  of  Defense  has 
come  a  long  way  in  improving  its  analytical  methods  for  planning 
under  uncertainty.  Today’s  DoD  planners  consider  a  broad  range  of 
challenges  represented  most  prominently  by  a  substantial  set  of  defense 
planning  scenarios  developed  as  part  of  what  is  called  the  “Analytic 
Agenda.”  The  scenarios  are  used  to  coordinate  planning  across  the 
entire  department,  as  well  as  to  encourage  appropriate  capability  devel¬ 
opments  (Bexheld,  2004).  Treatment  of  uncertainty  still  falls  well  short 
of  what  we  consider  desirable,  but  great  strides  have  been  made  since 
the  early  1990s,  when  the  enrichment  of  scenarios  began.  That  progress 
includes  an  increasingly  healthy  degree  of  consistency  between  pro¬ 
gramming  analysis  and  operational  analysis. 

Motivation  for  taking  the  uncertainty  aspects  of  planning  seri¬ 
ously  can  be  provided  by  a  few  examples. 

Who  Would  Have  Expected ?  Defense  planners  were  as  serious  in 
the  1980s  as  they  are  today,  and  some  considered  numerous  possible 
scenarios  for  the  future.  To  our  knowledge,  however,  none  predicted 
or  paid  much  attention  to  the  possibility  of,  e.g.,  (1)  a  campaign  of 
attempted  coercion  through  strategic  bombing  of  Serbia  (1998);  (2)  a 
U.S.  invasion  of  Afghanistan  that  would  include  projection  of  joint 
forces  for  vast  distances  and  Special  Operations  Forces  (SOF)  fighting 
alongside  tribesmen  and  calling  in  precision  fires;  or  (3)  a  U.S.  inva¬ 
sion  of  Iraq  for  the  purpose  of  regime  change,  which  would  become  a 
manpower-intensive  operation  of  intended  stabilization.14 

How  “Details”  About  Assumptions  Matter.  Historical  examples 
illustrate  the  importance  of  parameterizing  a  scenario  space  when  ana¬ 
lyzing  potential  needs  and  solutions.  Such  “details”  as  the  assumed 
warning  time  matter  enormously,  and  different  assumptions  lead  to 
different  conclusions  about  the  capabilities  needed.  When  Saddam 
Hussein  invaded  Kuwait  in  1990,  the  United  States  began  deployment 
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of  forces  a  week  after  D-Day,  rather  than  one  to  three  weeks  before 
D-Day,  as  is  assumed  in  standard  planning  scenarios.  One  conse¬ 
quence  was  that  airmen  arriving  in  Saudi  Arabia  found  themselves 
with  a  severe  shortage  of  air-to-ground  weapons  because  logistics  had 
been  based  on  the  official  planning  scenario. 

During  the  1990s,  U.S.  planning  scenarios  included  prominently 
the  possibility  of  a  new  war  with  Saddam  Hussein’s  Iraq.  However, 
planning  emphasized  defense  or  quick  reaction,  followed  by  a  restora¬ 
tion  of  boundaries  or  a  subsequent  counteroffensive.  It  did  not  consider 
the  regime-change,  nation-building,  or  lengthy  stabilization  operations 
that  were  subsequently  launched.  As  is  now  well  known,  U.S.  force 
structure  and  Army  ground-force  doctrine — although  well  suited  to 
the  earlier  planning  scenario — were  not  well  suited  to  the  stabilization 
phase. 

Ultimately,  the  lesson  is  that  a  name-level  scenario,  such  as  a  “war 
with  Country  X,”  with  a  single  set  of  assumptions  is  a  poor  basis  for 
planning,  even  about  a  possible  new  war  with  Country  X  (Davis,  1994, 
2002)!  Hence  the  need  for  parameterization  of  scenarios  and  what  we 
call  “exploratory  analysis.” 

Let  us  now  turn  to  the  issue  of  taking  a  broad  view  of  what  the 
factors  and  uncertainties  involve. 

Including  All  Relevant  Dimensions.  It  is  common  for  today’s  DoD 
leaders  to  emphasize  the  need  to  consider  DIME  and  PMESII  aspects 
of  scenarios.*  Interestingly,  it  is  something  with  which  at  least  some 
defense  analysts  have  long  been  acquainted.  Some  tangible  examples 
may  illustrate  how  important  emphasizing  these  aspects  has  been. 

Forward  Defense  and  Flexible  Response.  During  the  Cold  War, 
NATO’s  military  strategy  was  dominated  by  political  and  economic 
considerations.15  This  was  manifested  by  (1)  dependence  on  U.S.  stra¬ 
tegic  nuclear  forces  as  an  overarching  deterrent;  (2)  a  forward-defense 
posture  that  was  militarily  risky  but  essential  politically;  (3)  the  intro- 


Diplomatic,  information,  military,  and  economic  (DIME);  and  political,  military,  eco¬ 
nomic,  social,  infrastructure,  and  information  (PMESII).  The  usual  syntax  is  to  refer  to 
PMESII  factors  and  DIME  instruments.  It  is  unclear  that  much  value  has  been  added  by 
using  so  many  capital  letters. 
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duction  of  limited  nuclear  options  to  enhance  flexible  response,  which 
in  turn  improved  the  credibility  of  NATO’s  defense  generally;  and  (4) 
later  improvement  of  conventional  defenses  without  building  a  credible 
offensive  threat.  To  narrow  military  analysts,  the  political  factors  may 
have  been  seen  as  mere  constraints,  but  to  defense  planners,  they  were 
central.  Influences  also  worked  in  the  other  direction.  In  particular, 
during  the  latter  days  of  the  Cold  War,  military  analysis  led  the  United 
States  to  conclude — and  to  convince  its  NATO  allies — that  conven¬ 
tional  arms  control  should  proceed  toward  equal  limits,  rather  than  a 
scaling  down  of  existing  forces.16  Although  diplomats  expected  this  to 
be  difficult  to  negotiate,  there  was  an  iron  logic  militarily:  Proportional 
cuts  would  disproportionately  endanger  NATO’s  security.  Even  later, 
analyses  of  nonstandard  scenarios  demonstrated  that  the  treatment  of 
conventional  forces  in  Europe  should  focus  on  a  kind  of  operational 
arms  control  constraining  real-world  feasibility  of  a  Soviet/Pact  short- 
mobilization  attack,  rather  than  on  such  alleged  confidence-building 
measures  as  withdrawal  from  borders.17  The  NATO  Cold  War  expe¬ 
rience,  then,  was  a  constant  exercise  in  PMESII  factors  and  DIME 
instruments. 

The  Messy  Nature  of  Modem  War.  The  web  of  relationships  con¬ 
necting  political,  economic,  and  military  factors  in  modern  warfare  is 
well  described  by  General  Wesley  Clark,  based  on  his  experiences  as 
Combatant  Commander  during  the  Balkan  crises  of  the  late  1990s 
(Clark,  2001).  In  the  Kosovo  crisis,  for  example,  NATO  sought  to  use 
strategic  bombing  to  coerce  President  Milosevic  into  capitulating.  This 
bombing  was  constrained  by  alliance  disagreements,  complex  com¬ 
mand  and  control,  and  other  factors.  Political  considerations  (and  dif¬ 
ferences  of  opinion  within  the  military  community)  seemed  to  rule  out 
a  ground-force  option  for  many  weeks,  even  though  such  an  option 
could  scarcely  have  failed  to  alleviate  coercive  pressures.  Then,  after  a 
surprisingly  long  time,  but  before  the  ground  option  had  been  resur¬ 
rected  and  planned,  Milosevic  did  indeed  quit.18  Sorting  out  how  the 
various  factors  contribute  is  difficult  in  “messy”  modern  war. 

The  Concept  of  a  Spanning  Set  in  Capabilities-Based  Planning. 
Capabilities-based  planning  (CBP)  was  mandated  by  the  2001  Qua¬ 
drennial  Defense  Review,  which  urged  planners  to  focus  more  on  the 
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attributes  of  capability  that  may  be  needed  than  on  particular  threat 
scenarios.  Background  analysis  should  consider  a  wide  range  of  both 
name-level  scenarios  and  key  factors  within  them  (what  we  refer  to  in 
this  monograph  as  “parameters”).19  This  has  been  the  intention  in  the 
work  of  DoD’s  Analytic  Agenda,  something  emphasized  in  guidance 
from  OSD  (Policy).20 

The  Value  of  Concrete  Scenarios.  Despite  the  need  to  think 
broadly  about  the  entire  scenario  space  of  possibilities,  there  is  great 
value  in  considering  particular  scenarios  of  crisis  and  war  in  which  a 
great  deal  of  context  is  specified.  The  reasons  include 

•  Helping  the  analysts  and  the  military  operators  who  work  with 
them  “get  into”  the  problem.  Concreteness  is  both  motivational 
and  informative. 

•  Helping  to  make  assumptions  explicit  and  meaningful. 

•  Helping  decisionmakers  assess  whether  the  capability  at  issue 
would  merely  be  nice  to  have  logically  or  would  make  a  differ¬ 
ence  in  plausible  cases  of  concern  (capabilities-based  planning  was 
never  intended  to  be  a  blank  check). 

•  Reducing  the  burden  on  analysts  and  analytic  organizations: 
In  the  current-day  Pentagon,  much  effort  goes  into  imposing 
common  scenarios  and  working  them  through  in  detail.  That  is 
not  possible  if  the  number  of  scenarios  is  large,  because  the  spe¬ 
cially  capable  officers  who  must  work  through  force- employment 
concepts  become  overburdened. 

•  Facilitating  communication:  It  is  easier  to  compare  options  in  a 
strategic-level  framework  if  everyone  has  a  fairly  concrete  mental 
image  of  what  the  evaluation  cases  are  all  about. 

The  Search  for  a  Spanning  Set.  Accepting  that  it  is  valuable  to 
have  specific  scenarios  at  some  point,  the  question  then  arises  as  to  how 
many  and  what  type  of  scenarios  are  considered  and  how  one  knows 
that  enough  scenarios  have  been  considered.  That  is,  which  scenarios, 
with  what  characteristics ?  And  why  are  those  chosen ?  Such  matters  are 
seldom  discussed  with  any  rigor.  Therefore,  we  use  the  concept  of  a 
spanning  set: 
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A  spanning  set  is  a  set  of  test  scenarios  chosen  so  that  if  alter¬ 
native  proposed  systems  are  tested  against  them,  the  systems 
will  be  “stressed”  in  all  the  appropriate  ways.  Systems  that 
do  well  across  these  test  cases  should  do  well  in  the  situations 
that  arise  in  the  real  world. 

As  mentioned  earlier,  “requirements”  for  capability  are  best  seen 
as  the  outputs  of  a  decision  process  informed  by  first-order  analysis.  To 
the  extent  that  planning  scenarios  are  used  to  assess  the  adequacy  of 
programs,  they  serve  as  something  like  requirements  and  should  reflect 
analysis  concluding  that  the  scenarios  are  appropriate  for  the  purpose 
being  considered.  That  is,  the  set  of  planning  scenarios  should  be  a 
spanning  set. 

Design  engineers  understand  such  concepts.  They  conceive  their 
design  space  analytically  and  parameterize  the  principal  variables — 
setting  up  models  in  which  those  key  variables  can  be  readily  varied. 
They  then  identify  distinguishable  regions  of  the  design  space,  such  as 
a  region  requiring  supersonic  aircraft  speeds  or  a  region  of  low  tem¬ 
peratures  for  a  laptop  computer.  Ultimately,  choices  must  be  made, 
and  some  of  the  possibilities  will  be  forgone.  Once  major  decisions  are 
made  by  the  client,  engineers  can  narrow  the  envelope  within  which 
the  system  will  perform.  This  is  still  very  different  from  designing  for 
“point  capability.”  For  example,  an  aircraft  may  be  required  to  oper¬ 
ate  with  a  high  degree  of  stability  and  performance  anywhere  within  a 
specified  envelope  of  altitude  versus  speed.  All  bets  are  off,  however,  for 
aircraft  flying  outside  the  envelope. 

Applying  the  same  logic  to  defense  planning  leads  to  a  process 
suggested  by  Figure  3.2.  The  first  task  is  to  think  broadly  about  the 
design  space  (the  space  of  possible  operational  contexts),  characteriz¬ 
ing  analytical  regions  that  are  distinct.  Policymakers  must  then  decide 
which  regions  of  capability  are  to  be  pursued,  a  decision  informed 
by  importance,  costs,  and  feasibility.  Given  such  a  decision  (perhaps 
recommended  to  the  SecDef  by  DoD’s  Deputies  Advisory  Working 
Group),  the  next  step  is  to  identify  scenarios  representative  of  each  of 
the  regions  for  which  capability  is  to  be  pursued.  These  become  de  facto 
requirements.  As  a  classic  example,  a  series  of  defense  secretaries  since 
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Figure  3.2 

Establishing  Priorities  in  a  Design  Space  of  Capability 
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the  1960s  have  required  strategic  mobility  capabilities  adequate  for  two 
major  “concurrent”  conflicts,  but  not  two  fully  simultaneous  conflicts. 
Why?  The  secretaries  were  presumably  convinced  that  the  price  tag 
would  be  high,  that  such  situations  were  unlikely,  and  that — even  if  a 
situation  of  simultaneous  wars  arose — the  United  States  would  prob¬ 
ably  proceed  more  or  less  sequentially. 

To  reinforce  the  point,  note  that  in  Figure  3.2,  analysis  occurs 
early,  to  inform  initial  decisions  about  what  test  scenarios  (and  the  per¬ 
formance  demanded  in  those  test  cases)  to  insist  upon.  Such  judgments 
should  not  be  based  on  intuition  alone. 

Define  CONOPS  and  Critical  Components  for  Options 

Concepts  of  Operations.  Evaluation  of  options  depends  upon 
having  a  suitable  concept  of  operations  (CONOPS).  If  a  new  weapon 
capability  is  evaluated  using  only  preexisting  CONOPS,  predicted 
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effectiveness  may  be  only  marginally  better  than  what  existed  previously. 
The  issue  of  CONOPS  becomes  particularly  important  when  assess¬ 
ing  options  that  are  significantly  different  from  predecessors21  or  when 
joint  CONOPS  offer  important  synergies.  The  analyst  must  be  wary  of 
two  different  types  of  potential  bias.  First,  scientists  and  engineers  are 
sometimes  prone  to  assume  synergies  that  may  not  be  realized  in  practice 
(e.g.,  that  the  vision  of  the  Global  Information  Grid  will  be  realized, 
with  everyone  getting  the  information  he  needs  when  he  needs  it). 
Second,  Service-oriented  operators  may  be  chary  of  assuming  efficient 
jointness  because  of  conservatism  based  on  history  and  their  own 
experiences.22 

Critical  Components  of  Capability.  An  operational  capability  exists 
only  if  all  the  critical  components  for  actually  executing  the  mission  in 
question  are  present.  Although  this  is  mere  tautology  (absence  of  a  crit¬ 
ical  capability  means  failure),  it  is  a  problem  in  the  real  world,  where 
programs  often  do  not  include  some  necessary  components — perhaps  to 
avoid  price  “sticker  shock”  and  with  the  rationalization  that  other  com¬ 
ponents  can  be  obtained  later.  The  missing  components  may  include, 
e.g.,  a  communication  link  to  other  C4ISR  (command,  control,  commu¬ 
nications,  computer,  intelligence,  surveillance,  and  reconnaissance)  sys¬ 
tems,  adequate  weapon  stocks,  logistical  equipment,  doctrine,  or  trained 
personnel.  DoD  recognizes  this  to  some  extent  by  referring  frequently  to 
DOTMLPF,  i.e.,  to  the  necessity  of  dealing  with  doctrine,  organiza¬ 
tion,  training,  materiel,  leadership  and  education,  personnel,  and  facili¬ 
ties.  One  of  the  strengths  of  U.S.  military  staff  operations  is  that  officers 
are  trained  to  worry  about  all  such  issues,  often  with  strategies-to-tasks 
breakdowns.  Although  it  does  not  extend  into  the  full  range  of  DOT¬ 
MLPF  issues,  DoD’s  concept  of  mission-capability  packages  (MCPs)  is 
similar  in  spirit  and  very  important  in  systems-engineering  work.23 

Assuring  the  presence  of  all  critical  components  is  nontrivial 
for  capabilities  analysis,  because  many  modeling  and  analysis  meth¬ 
ods  either  assume  uncritically  that  “other”  parts  of  the  system  exist  or 
employ  methods  such  as  linear  weighted  sums  to  characterize  net  capa¬ 
bilities  arising  from  a  mixture  of  strong  and  weak  components.24  As  we 
demonstrate  in  Chapter  Five,  it  is  possible  to  do  much  better. 
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Generate  and  Screen  Candidate  Options 

Although  options  are  typically  already  on  the  table  when  something 
like  a  capability-area  review  begins,  those  options  are  often  insuffi¬ 
ciently  comprehensive  or  somehow  parochial.  One  task  for  analysis, 
then,  is  to  generate  a  more  complete  set  of  options  that  might  be  con¬ 
sidered  and  to  then  conduct  a  preliminary  screening  to  identify  the 
candidate  options  worthy  of  fuller  study  within  a  portfolio-analysis 
framework.  We  shall  discuss  mechanisms  for  doing  so  more  fully  in 
Chapters  Four  and  Five. 

Evaluate  the  Options  in  DIME-Sensitive  Portfolio  Analysis 

The  Value  of  Portfolio  Analysis.  It  is  one  thing  for  potential  capa¬ 
bilities  to  appear  very  useful  for  ideal  cases  defined  technically;  it  is 
quite  another  for  those  concepts  to  make  sense  operationally,  economi¬ 
cally,  and  strategically.  Therefore,  the  analytic  framework  should  give 
all  of  the  dimensions  visibility.  Consistent  with  the  need  to  plan  under 
deep  uncertainty,  the  framework  should  help  decisionmakers  under¬ 
stand  both  where  the  proposed  capability  would  work  and  where  it 
would  not.  It  should  address  risks  and  also  upside  opportunities  that 
would  be  bonuses.  More  generally,  analysis  should  give  decisionmakers 
a  sense  of  the  degree  to  which  their  investments  are  balanced  across  a 
wide  range  of  considerations.  This  is  precisely  what  portfolio  analysis 
is  good  for,  and  we  recommend  that  it  be  used  routinely.  We  shall 
describe  what  we  have  in  mind  in  more  detail  in  Chapter  Four  and 
then  illustrate  it  in  Chapters  Five  and  Six. 

The  Issue  of  Metrics.  One  aspect  of  evaluation  is  finding  appro¬ 
priate  metrics  by  which  to  assess  effectiveness,  efficiency,  and  progress. 
A  subtle  problem  here  is  that  the  framework  and  metrics  need  to  fit  the 
problem  area  at  a  technical  and  operational  level,  rather  than  merely 
seeming  reasonable  to  strategic  thinkers  distant  from  the  technical  and 
operational  issues.  Even  well-chosen  top-level  objectives,  such  as  those 
relating  to  agility,  must  be  translated  into  more-concrete  and  measur¬ 
able  factors.25 

Historically,  discussions  of  evaluation  have  emphasized  the  alleged 
necessity  of  defining  quantitative  metrics.  That  can  be  counterproduc¬ 
tive.  When  carried  to  the  extreme,  it  results  in  defining  metrics  that 
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are  conveniently  quantitative  and  measurable  but  off  point.  For  exam¬ 
ple,  a  combat  unit  may  have  equipment  that  makes  it  physically  agile, 
but  it  may  lack  the  doctrine,  cultural  awareness,  and  specialized  train¬ 
ing  enabling  it  to  be  effectively  agile  in  a  difficult  environment  such 
as  today’s  Iraq.  Once  this  is  acknowledged,  of  course,  metrics  can  be 
introduced  that  measure  time  spent  in  relevant  operations  or  training. 
Even  then,  however,  a  commander’s  soft  judgments  may  be  necessary 
to  assess  agility  realistically.  The  commander  might  know,  for  example, 
that  the  unit’s  past  experience  or  training  taught  the  wrong  lessons. 

Characterize  Shortfalls,  Opportunities,  and  Surpluses 

Shortfalls.  Assuming  that  capabilities  in  a  particular  area  can  be 
characterized  analytically,  the  next  task  is  to  characterize  shortfalls  of 
projected  capability.  It  is  already  common  throughout  DoD  and  the 
Services  to  have  presentations  that  do  so.  These  may  be  the  result  of  off¬ 
site  meetings  in  which  military  officers  take  a  hard  look  at  their  current 
and  projected  operational  capabilities  and  make  subjective  assessments 
about  areas  of  relative  strength  and  weakness. 

It  is  less  common  to  develop  such  assessments  with  much  rigor. 
Some  of  the  shortfalls  identified  may  be  spurious  if,  for  example,  they 
are  based  on  analysis  that  ignores  qualitative  considerations  such  as 
the  fighting  quality  of  an  army  or  the  disparity  in  command  and  con¬ 
trol  between  two  air  forces.  Other  shortfalls  may  be  underestimated 
because  of  excessive  optimism  regarding  warning,  allies,  and  the  like. 

As  discussed  in  the  previous  section,  capabilities-based  planning 
considers  a  wide  range  of  plausible  scenarios  and  operational  circum¬ 
stances.  This  is  not  really  straightforward,  because  an  infinite  number 
of  possibilities  exist  and  because  subjective  judgments — especially 
unaided  ones — are  often  poor  at  characterizing  results  across  the  pos¬ 
sibility  space. 

Opportunities.  Identifying  shortfalls  is  common  and  expected, 
but  it  is  also  important  to  identify  opportunities  and  surpluses.  For 
example,  the  United  States  recognized  in  the  early  1970s  that  it  had 
the  opportunity  to  render  Soviet  air  defenses  obsolete  with  stealth  tech¬ 
nology.  Doing  so  would  not  only  be  militarily  effective,  it  would  also 
be  a  U.S. -style  asymmetric  strategy,  one  using  technology  to  undercut 
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the  value  of  something  on  which  the  adversary  (the  Soviet  Union)  had 
spent  the  equivalent  of  hundreds  of  billions  of  dollars.26  Although  pen¬ 
etrating  defenses  with  traditional  means  would  probably  have  remained 
possible,  albeit  with  increasing  dependence  on  electronic  countermea¬ 
sures,  stealth  provided  a  leapfrog  opportunity. 

Another  well-known  example  was  the  opportunity  provided 
by  precision-guided  munitions  (PGMs).  Again,  the  military  value  of 
weapon  accuracy  was  appreciated,  but  the  true  significance  was  greater: 
Soviet  military  leaders  were  extremely  concerned  about  the  prospect  of 
these  weapons  as  of  the  early  1980s,  because  they  recognized  that  the 
Soviet  industrial  base  was  unable  to  counter  such  development.  Soviet 
concerns  on  this  score  were  not  well  recognized  for  some  time  in  the 
West,  so  while  most  Western  commentators  on  the  conventional  mili¬ 
tary  balance  were  full  of  gloom  and  doom  because  of  Soviet  quantita¬ 
tive  advantages,  the  Soviets  themselves  saw  dark  days  ahead.27 

Surplus  Capability.  If  identifying  opportunities  is  somewhat 
unusual  in  capability  assessments,  identifying  excesses  is  extremely  so. 
It  is  simply  unnatural  for  organizations  to  find  their  resources  to  be 
more  than  adequate  for  the  tasks  assigned.  It  is  notoriously  difficult  for 
the  Secretary  of  Defense  to  find  a  Service,  or  even  a  branch  within  a 
Service,  willing  to  say,  “Oh,  we  could  tolerate  a  10  percent  cut  in  our 
budget;  it  would  not  significantly  affect  our  capability  to  do  our  job.” 

Efforts  have  been  made  in  modern  times  to  identify  excesses  in 
order  to  pay  for  other  initiatives,  such  as  the  transformation  initiatives 
of  Secretary  Rumsfeld.  Some  of  those  efforts  have  been  controversial, 
or  even  misguided.  In  the  late  1990s,  it  was  fairly  common  for  observ¬ 
ers  to  claim  that  the  U.S.  Army  could  be  cut  back  by  another  20  per¬ 
cent  because  of  the  changing  nature  of  warfare,  which  was  putting 
a  premium  on  air  power  and  precision  weapons.  The  Army’s  alleged 
“excess”  depended  sensitively  on  the  implicit  assumption  that  actual 
demands  would  be  comparable  to  those  of  planning  scenarios  in  which 
the  United  States  was  defeating  invading  armies  that  conveniently 
poured  down  narrow  highways,  making  themselves  easy  targets  for 
the  emerging  precision  fires.  Only  a  few  years  later,  with  the  bene¬ 
fit  of  experience  in  Afghanistan  and  Iraq,  it  is  obvious  that  there  are 
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too  few  U.S.  ground  forces  for  what  is  needed  in  manpower-intensive 
operations.28 

Implications.  The  moral  of  the  story  is  that  the  methodology  used 
to  identify  shortfalls,  opportunities,  and  excesses  needs  to  be  rigorous, 
broadly  based,  and  imaginative.  One  way  to  report  the  results  of  such 
work  is  to  address  both  upside  opportunities  and  downside  risks  explic¬ 
itly.  Indeed,  doing  so  is  arguably  an  example  of  best  practices  in  deci¬ 
sion  support,  one  with  roots  in  modern  decision  science.29  As  simple  as 
that  concept  may  be,  it  is  notably  absent  from  most  current-day  support 
of  DoD  decisionmaking  (although  the  decisionmakers  often  add  con¬ 
siderations  of  opportunities  and  risks  in  their  heads).  We  shall  discuss 
mechanisms  for  implementing  such  an  approach  in  later  chapters. 

Add  Options:  Remedies  and  Possible  Reductions 

Identifying  potential  remedies  is  a  routine  undertaking,  but  DoD  lead¬ 
ers  sometimes  worry  about  whether  the  remedies  considered  are  suffi¬ 
ciently  creative  and  farsighted.  In  some  cases,  the  issue  is  having  future 
operational  concepts  be  born  joint;  in  other  cases,  the  issues  relate  more 
to  using  technology  well  or  to  being  sufficiently  realistic  about  how 
proposed  remedies  would  fare  in  a  range  of  circumstances,  as  men¬ 
tioned  above. 

To  illustrate  the  issues  that  can  arise,  consider  ballistic-missile 
defense  of  the  U.S.  homeland,  allies,  and  forward-deployed  forces  (see 
Chapter  Six).  The  straightforward  remedies  that  are  most  comfortable 
for  existing  organizations  involve  long-range  ground-based  intercep¬ 
tors  and  ship-based  interceptors  for  medium-  and  intermediate-range 
attackers.  In  considering  remedies,  however,  the  Missile  Defense 
Agency  must  consider  a  diversity  of  conflicts,  potential  attack  modes  (to 
include  attacks  from  ships  relatively  close  to  the  United  States),  coun¬ 
termeasures,  and  geographies.  Questions  arise  about  the  possible  long¬ 
term  role  of  space-based  weapons  or  midterm  interceptors  launched 
from  bombers  or  even  fighter  aircraft.  These  types  of  remedy  would 
be  rather  unconventional  and  organizationally  disruptive;  they  might 
also  have  complex  international  ramifications.  Another  issue  is  how  to 
assure  that  the  future  ballistic-missile  defense  system  (BMDS)  exploits 
the  possibilities  of  networking,  including  the  Global  Information  Grid 
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and  subsystems  with  more-stressful  technical  requirements.  Will  it  be 
possible  to  do  effective  and  efficient  battle  management  drawing  upon 
whatever  BMDS  components  happen  to  be  deployed  in  relevant  but 
geographically  separated  ground  bases,  ships,  and  aircraft,  and  to  do 
so  using  both  U.S.  and  allied  systems?  And,  taking  a  broader  view 
yet,  what  about  aspects  of  the  “total  system,”  such  as  defending  U.S. 
installations  or  attacking  or  disrupting  those  of  the  adversary,  perhaps 
preemptively  and  perhaps  with  means  ranging  from  special  forces  to 
cyber  attack? 

Iterate 

As  indicated  in  Figure  3.1,  the  analytical  process  for  capability 
reviews  should  be  iterative:  Analysis  will  generate  ideas  for  alternative 
approaches,  which  will  then  need  to  be  analyzed  anew;  doing  so  will 
point  out  the  need  to  improve  the  analytic  framework  itself  to  make  it 
include  additional  considerations.  Iteration  should  continue  until  stable 
conclusions  emerge  or  the  time  available  is  exhausted.  Such  iterations 
are  important  in  practice  and  a  major  reason  for  high-quality  analyses 
taking  months  rather  than  weeks. 

Having  provided  an  overview  of  the  analytic  framework  that  we 
suggest,  we  next  discuss  some  of  the  tools  needed,  after  which  we  shall 
present  some  examples. 
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Tools  to  Enable  the  Framework 


A  number  of  types  of  models  and  tools  are  needed  to  accomplish  the 
goals  described  in  previous  chapters.  Figure  4.1  indicates  this  sche¬ 
matically.  The  arrows  are  deliberately  two-way.  It  would  be  folly  to 
construct  a  bottom-up  system  for  decision  support,  because  so  much 
of  the  reasoning  and  analysis  needs  to  be  top-down  in  character.  Nev¬ 
ertheless,  the  quality  of  the  framework  used  at  higher  levels  and  the 
validity  of  the  information  presented  there  sometimes  depend  on  deep 
knowledge,  such  as  that  residing  in  the  world  of  systems  engineering, 
detailed  modeling  and  simulation,  and  experimentation. 

Figure  4.1 

The  Role  of  Models  and  Portfolio  Tools  in  Investment  Analysis 
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Although  this  chapter  discusses  such  cross-level  issues  in  terms 
of  tools,  the  most  important  aspects  of  cross-level  work  are  typically 
organizational  and  social.  It  is  all  too  easy  for  those  who  do  the  rel¬ 
atively  higher-level  work  and  those  who  do  the  relatively  lower-level 
work  to  be  in  distinct  organizations  with  little  cross-boundary  commu¬ 
nication.  That  can  be  a  serious  problem.  Ironically,  despite  the  natural 
resistance  to  cross-boundary  communication  (the  specter  of  meetings 
and  bureaucracy  and  of  dealing  with  unfamiliar  people),  such  collabo¬ 
rations  can  be  very  interesting  and  rewarding.30 

Let  us  briefly  describe  the  elements  of  Figure  4.1,  one  by  one. 


Systems  Engineering,  Modeling,  Simulation,  and  Analysis, 
and  Experiments 

Although  merely  touched  upon  in  this  monograph  (see  also  Appendix 
B),  the  technical  foundations  for  capability  development  are  typically 
located  at  the  levels  of  systems  engineering  and  underlying  technology, 
including  experiments  to  motivate  and  test  concepts.  The  tools  used 
at  that  level  include  relatively  detailed  models  and  simulations.  These 
are  sometimes  just  tools  used  in  analysis  and  sometimes  the  reposi¬ 
tory  of  the  most  accurate  and  precise  knowledge  available.  Some  of  the 
detailed  simulations  are  used  directly  in  experimentation  and  merge 
to  a  greater  or  lesser  degree  with  components  of  real-world  command 
and  control  systems.31  As  a  practical  matter,  experimentation  programs 
are  often  more  useful  for  testing  the  simulations  than  for  establishing 
a  firm  body  of  empirical  data;  many  decisions  about  design,  develop¬ 
ment,  and  production  are  primarily  dependent  on  information  from 
those  simulations. 

Virtually  all  acquisition  projects  have  associated  systems- 
engineering  models,  although  many  of  them  reside  only  in  industry. 


Capability  Models 

By  “capability  model”  we  mean  a  model  that  characterizes  the  effec¬ 
tiveness  of  a  current  or  prospective  system  or  collection  of  forces  in 
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situations  of  interest.  Although  there  is  overlap  among  levels,  these 
models  are  typically  somewhat  higher-level  (and  more  aggregated)  than 
those  for  systems  engineering.  Many  examples  exist  throughout  the 
defense  community.32  For  example,  EADSIM  describes  effectiveness 
of  air-defense  systems  in  diverse  scenarios  of  enemy  attack.  RAND’s 
CAMMD  (Capabilities  Analysis  Model  for  Missile  Defense)  is  a  high- 
level  model  developed  for  the  Missile  Defense  Agency  that  describes 
the  effectiveness  of  a  Ballistic-Missile  Defense  System  (BMDS)  as  a 
function  of  the  characteristics  of  the  system  and  the  attack.  Two  earlier 
RAND  models  (EXHALT  and  EXHALT-CF)  describe  effectiveness 
in  interdiction  campaigns  as  a  function  of  forward  presence,  warning, 
access  problems,  the  time  required  for  suppression  of  air  defenses,  and 
other  factors.  Such  models  are  typically  math-  and  physics-based  and 
parametric.  For  example,  the  model  used  in  Chapter  Four  to  estimate 
effectiveness  of  tactical  air  forces  or  long-range  bombers  has  as  a  key 
input  the  class  of  air  defenses  the  aircraft  would  encounter.  A  data  table 
translates  that  class  into  probability  of  penetration.  Campaign  models 
(e.g.,  JICM,  Thunder,  and  ITEM)  are  special,  rather  highly  aggregated 
capability  models.  The  “capabilities”  treated  by  such  models  relate  more 
to  integrated,  aggregate-level  force  capabilities  than  to  the  capabilities 
crucial  in  given  missions.  Campaign  models  are  discussed  in  a  recent 
master-plan  study  for  DoD’s  Modeling  and  Simulation  Coordination 
Office.33 

Capability  models  exist  for  almost  all  subjects  of  interest  in 
defense-oriented  capabilities-based  planning.  The  current  models,  how¬ 
ever,  may  not  be  especially  suitable  for  higher-level  tradeoffs.  For  these, 
lower-resolution  models  tend  to  be  more  valuable  than  high-fidelity 
models  because  (1)  they  have  fewer  parameters  about  which  to  reason, 
(2)  setup  time  is  shorter,  and  (3)  decisionmakers  need  explanations  of 
important  conclusions  and  often  prefer  them  to  be  simple  and  under¬ 
standable,  rather  than  dependent  on  a  myriad  of  unstated  assumptions. 
Ideally,  each  capability  area  will  have  a  family  of  capability  models  on 
which  to  draw,  each  with  different  strengths  (e.g.,  greater  or  lesser  reso¬ 
lution  in  various  respects,  different  perspectives),  but  all  informed  by 
the  others.  Although  somewhat  unusual  at  present,  such  model  fami¬ 
lies  are  feasible  and  highly  desirable.34 
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Generating  and  Screening  Candidate  Options 

It  is  often  assumed  in  discussions  of  methodology  that  the  options  to 
be  evaluated  have  already  been  identified.  However,  the  options  that 
arise  for  consideration  in  a  given  capability  area  will  often  come  from 
people  and  organizations  who  developed  them  based  on  their  organiza¬ 
tions’  past  efforts,  knowledge,  and  interests.  The  suggested  options  will 
thus  reflect  diverse  assumptions  about  what  capabilities  are  needed. 
The  individuals  involved,  in  many  cases,  will  not  have  thought  much 
about  opportunities  for  synergy,  either  across  Services  or  across  capa¬ 
bility  areas,  except  where  synergy  is  natural  for  their  particular  inter¬ 
ests  (e.g.,  an  airplane  builder  will  see  multiple  missions).  Further,  only 
sometimes  will  the  individuals  have  offered  up  variants  that  cost  and 
deliver  more  or  less  than  what  they  recommend.  As  a  result,  those 
who  must  allocate  limited  resources  often  lack  information  they  need 
for  tradeoff  analyses,  for  combined  strategies  exploiting  synergies  and 
hedging  against  risks,  and  for  making  program  adjustments  wisely 
(e.g.,  increasing  or  decreasing  allotments  to  various  programs,  relative 
to  what  is  requested).  Thus,  there  is  need  for  a  more  comprehensive 
and  systematic  approach  to  option-generation,  rather  then  merely  the 
evaluation  of  options  being  proposed  in  the  usual  manner. 

As  described  in  a  companion  RAND  report  (Davis,  Shaver, 
Gvineria,  and  Beck,  2007),  we  have  developed  the  Building  Blocks 
to  Composite  Options  Tool  (BCOT)  to  help  in  developing  candidate 
options  worthy  of  more-detailed  analysis  in  a  full  portfolio  structure. 
BCOT  is  discussed  further  below  and  is  described  schematically  in 
Figure  4.2. 

In  the  experiments  that  we  have  conducted,  we  began  with  10 
to  15  building  blocks,  generated  thousands  of  composite  options,  and 
then  used  the  screening  methods  to  reduce  the  list  to  about  five  to  20 
options. 

Analysis  begins  by  constructing  a  list  of  building-block  invest¬ 
ment  options,  such  as  buying  a  new  radar,  kill  vehicle,  or  upgrade  to  a 
missile.  Composite  options  are  then  constructed  for  all  combinations 
of  those  building  blocks.  The  composite  options  are  next  analyzed  with 
a  screening  tool  to  eliminate  those  that  don’t  meet  thresholds  of  effec- 
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Figure  4.2 
Schematic  of  BCOT 
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tiveness,  that  exceed  a  threshold  of  cost,  or  that  are  clearly  inferior  to 
other  options  from  a  cost-effectiveness  perspective.  The  methodology 
expands  significantly  upon  mathematical  techniques  developed  years 
ago.  That  earlier  work  found  options  on  the  so-called  efficient frontier .*35 
Ours  finds  non-redundant  options  that  are  on  or  close  to  the  efficient 
frontier  (see  Figure  4.3).  Redundancy  here  refers  to  an  option  having 
building  blocks  that  add  nothing  to  effectiveness  but  that  do  add  costs. 
Retaining  options  that  are  close  to  the  efficient  frontier  is  important, 
because  screening  could  otherwise  discard  some  that  appear  in  a  first- 
order  analysis  to  be  inferior  but  that  would  be  superior  when  viewed  in 
a  fuller  portfolio  analysis. 


Portfolio-Analysis  Tools 

While  the  capability  model  produces  raw  effectiveness  information  as 
a  function  of  many  factors,  it  is  the  job  of  the  portfolio-analysis  tool  to 
help  make  sense  of  the  alternatives  for  investment.  The  user  of  such  a 


The  efficient  frontier  connects  Pareto-optimal  points  (Winston,  1994).  In  a  plot  of  effec¬ 
tiveness  versus  cost,  a  Pareto-optimal  point  is  at  least  as  effective  as  any  other  point  with  the 


same  cost. 
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Figure  4.3 

Schematic  Depiction  of  Finding  Points  Near  the  Efficient  Frontier 
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tool  must  decide  on,  e.g.,  the  investment  options  to  be  compared,  the 
multifaceted  measures  of  effectiveness  to  be  applied,  and  measures  of 
risk. 

Once  these  are  decided,  the  structure  of  the  analysis  is  determined, 
but  it  is  then  necessary  to  populate  that  structure,  i.e.,  to  provide  data 
on  how  the  various  options  fare  under  the  different  measures  of  effec¬ 
tiveness,  risk,  and  so  on,  and  how  much  the  options  cost.  The  terminol¬ 
ogy  of  portfolio  analysis  comes  from  the  use  of  somewhat  analogous 
methods  in  economic  theory.  An  investor,  for  example,  should  main¬ 
tain  a  suitable  portfolio  of  investments  of  different  types  (e.g.,  stocks, 
bonds,  real  estate,  gold),  in  order  to  be  well  hedged  against  various 
types  of  risk.  The  investor  should  review  and  adjust  that  portfolio  from 
time  to  time  to  assure  that  objectives  and  risks  are  properly  balanced. 

The  portfolio-analysis  tool  should  generate  tables,  graphics,  and 
text — output  that  will  help  decisionmakers  assess  the  relative  strengths 
and  weaknesses  of  the  options  and  make  judgments  such  as  what 
to  buy  first,  and  so  on.  These  outputs  may  take  the  form  of  colorful 
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“scorecards,”  cost-effectiveness  graphics,  pairwise-comparison  tables  or 
charts,  or  other  displays. 

Portfolio-analysis  tools  have  existed  for  many  years;  some  are 
custom-built  by  the  users,  and  some  have  been  developed  commer¬ 
cially  for  routinized  analyses  such  as  arise  in  the  commercial  world. 
The  relatively  simple  Balanced  Scorecard  method  has  been  much  dis¬ 
cussed  in  recent  years  (Kaplan  and  Norton,  1996).  A  number  of  tools 
draw  on  the  theoretical  business  literature,  such  as  the  work  of  Peter 
Weill  on  information  technology  portfolios.  Providing  such  tools  is  a 
large  business  in  itself.  The  tools  needed  for  DoD’s  purposes,  however, 
tend  to  be  more  complicated  and  sophisticated  in  some  respects,  multi¬ 
faceted  (there  is  no  single  measure  of  goodness,  such  as  expected  profit, 
in  military  applications),  and  useful  for  characterizing  diverse  types  of 
risk  and  opportunity.  Thus,  portfolio  analysis  in  defense  work  should 
be  quite  different  from  that  in  the  financial  world  (Davis,  Gompert, 
and  Kugler,  1996). 

Two  related  tools  have  been  developed  at  RAND  over  the  past 
decade,  both  of  which  are  in  use  currently.  Development  of  these  tools 
was  strongly  motivated  by  our  research  in  modern  decision  science 
(Davis,  Kulick,  and  Egner,  2005),  as  well  as  many  years  of  experience 
with  studies  in  support  of  DoD  and  Service  decisionmakers. 

The  first  tool  is  DynaRank,  developed  primarily  to  rank  options 
to  assist  in  deciding  what  to  fund  (or  cut)  first,  second,  and  so  on  when 
trying  to  work  within  a  budget  (Hillestad  and  Davis,  1998). 36 The  most 
recent  tool  is  PAT  (Portfolio-Analysis  Tool),  a  generic  version  of  an  ear¬ 
lier  tool  developed  for  the  specialized  purposes  of  the  Missile  Defense 
Agency  (Dreyer  and  Davis,  2005).  Use  of  PAT  is  illustrated  in  the  next 
two  chapters.  At  a  technical  level,  some  of  PAT’s  more  important  fea¬ 
tures  are 

1.  Scorecard  methods  that  support  making  evaluations  across 
numerous  dimensions  of  effectiveness,  risk,  and  cost 

2.  Zoom  (drill-down)  for  visual  explanation 

3.  Nonlinear  assessment  methods 

4.  Alternative  “perspectives”  (assumption  sets  used  in  the  assess¬ 
ments)  by  which  to  perform  cost-benefit  calculations 
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5 .  Multiple  resolution,  enabling  one  to  enter  assumptions  at  differ¬ 
ent  levels  of  detail. 

Figure  4.4  indicates  schematically  the  way  PAT  operates  from 
a  top-down  perspective.  Discussion  begins  by  looking  at  a  summary 
scorecard.  A  request  for  explanation  (e.g.,  “Why  is  option  1  so  much 
better  than  option  2  as  judged  by  effectiveness  for  scenario  class  B?”) 
can  be  answered  by  zooming  into  a  second  level  of  detail  for  the  evalu¬ 
ation  of  scenario  class  B.  If  that  level  of  detail  is  insufficient,  another 
level  of  zoom  is  possible.  And,  where  appropriate,  it  is  possible  to 
zoom  to  an  even  deeper  level,  what  we  call  the  level  of  systems  analy¬ 
sis,  to  see  crucial  charts  or  tables.  This  level  might  have  charts  with- 
families  of  curves  showing  how  results  vary  with  the  assumed  values 
of  key  parameters.  Or  one  level  might  have  a  live  model  (embedded 

Figure  4.4 

Hierarchy  of  Detail  in  RAND's  Portfolio-Analysis  Tool 
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within  Excel  or  separate)  that  allows  the  users  to  “see”  effects  of  changes. 
We  do  not  anticipate  that  there  will  be  many  decision  meetings  with 
live  tools  or  with  much  discussion  of  technical  detail,  However,  orga¬ 
nizing  the  information  in  this  way  makes  it  possible  to  tailor  presenta¬ 
tions  easily  and — in  response  to  either  careful  staff  review  or  a  deci¬ 
sionmaker’s  spot-check  questioning — to  drop  down  to  details  quickly. 


Tailoring  Analysis  for  Different  Contexts 

Good  tools  allow  tailoring  of  analytic  material  for  different  audiences 
and  occasions.  The  key  factors  in  such  a  tailoring  are  (1)  the  nature  of 
the  decisions  or  judgments  to  be  made;  (2)  the  decisionmakers’  relative 
interest  in  strategic,  technical,  or  process  issues;  (3)  the  decisionmakers’ 
preexisting  depth  of  knowledge;  (4)  time;  (5)  the  psychological  context 
resulting  from  other  contemporary  events;  (6)  the  format  of  presen¬ 
tation  (briefing,  discussion,  written  monograph,  or  a  combination  of 
these);  and  (7)  personal  inclinations  and  styles. 

Generalizations  on  these  matters  are  often  misleading.  Over  the 
years,  we  have  reported  results  of  studies  to  senior  people  in  meetings 
that  have  lasted  tens  of  minutes  or  hours.  We  have  gone  into  meetings 
expecting  a  high  level  of  interest  only  to  find  the  official  distracted, 
and  into  meetings  expecting  a  perfunctory  discussion  only  to  have  the 
official  engage  in  in-depth  questioning.  Most  commonly,  senior  clients 
have  been  briefed  and  have  done  little,  if  any,  reading  on  the  subject. 
However,  in  some  cases  they  have  done  significant  reading — usually 
of  executive  summaries,  but  sometimes  even  portions  of  the  full  text. 
Sometimes  the  reading  was  scheduled,  and  sometimes  it  was  ad  hoc, 
perhaps  on  an  airplane. 

Another  issue  on  which  generalization  is  inappropriate  concerns 
discussions  of  risk  and  uncertainty.  Some  high  officials  are  exceed¬ 
ingly  interested  in  these  matters,  regarding  them  as  at  the  core  of  their 
responsibilities.  They  wish  to  know  not  only  about  best  estimates,  but 
also  about  “What  ifs?”  Other  officials  are  more  interested  in  best  esti¬ 
mates  concerning  the  way  ahead  and  have  little  patience  for  discussion 
of  uncertainty,  which  they  see  as  cluttering  the  landscape  with  equivo¬ 
cation  and  tail-covering.  They  may  appreciate  risk  and  uncertainty  at 
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one  level  but  believe  that  the  only  sensible  way  to  proceed  is  to  go  with 
the  most  likely  option,  adapting  later  as  necessary.  Or,  in  a  variant, 
they  may  believe  that  it  is  better  to  just  press  ahead  and  do  something 
sensible  (and  iterate  later)  than  to  agonize  about  imperfectly  under¬ 
stood  alternatives. 

Both  history  and  decisionmaking  research  tell  us  that  both  styles 
exist  widely  among  even  very  good  decisionmakers.  Further,  analytical 
decisionmakers  sometimes  are  poor  at  strategy  and  miss  common-sense 
points,  whereas  intuitive  decisionmakers  are  sometimes  prescient  and 
effective  and  sometimes  disastrously  wrong.  Both  types  (and  the  many 
hybrids)  can  benefit  greatly  from  good  staff  processes  and  staff  analysis, 
but  how  they  can  best  be  helped  varies  substantially.  These  and  other 
issues  are  discussed  in  related  RAND  work  on  modern  decision  science 
(Davis,  Kulick,  and  Egner,  2005;  Davis  and  Kahan,  2007). 

A  basic  question  we  addressed  as  we  prepared  this  monograph  was 
how  to  deal  with  the  variations  described  above.  As  strategic  analysts, 
we  are  naturally  very  interested  in  methodology  and  subtleties,  but  in 
our  role  as  advisors  on  support  for  decisionmakers,  we  must  be  realistic 
about  what  is  both  needed  and  useful. 

To  the  extent  that  we  have  a  solution,  we  believe  that  it  depends 
on  adhering  to  the  principles  summarized  in  Table  4.1.  That,  in  turn, 
means  relying  on  first-rate  analysts  operating  in  circumstances  that 
allow  them  to  do  the  kind  of  work  described. 

Having  provided  an  overview  of  both  methodology  and  tools,  we 
shall  now  provide  two  examples  of  how  they  apply  to  concrete  prob¬ 
lems.  Chapter  Five  deals  with  the  issue  of  Global  Strike  and  Chapter 
Six  deals  with  Ballistic  Missile  Defense. 
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Table  4.1 

Principles  for  Tailorable  Decision  Support 


Principle 

Comment 

Prepare  material  with  structure  and 
rigor. 

Do  so  even  if  only  highlights  are  likely  to  be 
presented  to  decisionmakers. 

Bring  to  bear  imagination,  creativity, 
and  cross-cutting  thinking. 

Create  frameworks  that  deal  with  risk, 
uncertainty,  and  political-military 

This  is  in  contrast  to  "by  the  numbers" 
analysis  and  bean  counting. 

Tailoring  to  the  context  is  essential. 

content. 

Insist  on  candor  and  clarity  about  both  Do  so  even  if  the  audience  may  react 
upsides  and  downsides;  reveal,  rather  negatively  to  reminders  of  risk  or  bad 


than  hide  risks. 

Develop  layered  versions  of  analyses 
and  stories. 

Develop  different  layerings  for 
different  contexts. 

news. 

Each  layer  should  be  as  self-contained  and 
comprehensible  as  possible. 

The  decompositions  appropriate  for  20-year 
strategic  planning,  "right-now"  budget 
decisions,  and  concept  decision  reviews  are 
fundamentally  different. 

Whether  supporting  analytical  or 

Single  words,  such  as  "adaptiveness"  or 

naturalistic  decisionmakers,  present  "agility"  are  often  used  to  convey  the 
material  so  as  to  encourage  and  assist  same  intention  as  FARness. 
planning  for  flexibility,  adaptiveness, 
and  robustness  (FARness). 


CHAPTER  FIVE 


A  Notional  Example:  Global  Strike 


In  this  chapter,  we  walk  through  a  notional  application  of  our  method¬ 
ology  to  the  Global  Strike  problem.  When  we  chose  Global  Strike  for 
our  application,  it  had  not  yet  been  defined  as  one  of  AT&L’s  capabil¬ 
ity  areas,  and  no  major  studies,  such  as  analysis  of  alternatives  (AoA), 
had  yet  been  conducted.  We  saw  this  as  an  advantage  for  what  was 
intended  to  be  an  unclassified  illustration  of  methodology.  Later,  we 
were  asked  by  AT&L  to  develop  the  work  further  as  preparation  for 
an  evaluation  of  alternatives  (EoA)  for  Global  Strike  within  the  new 
Concept  Decision  Process  (Krieg,  2007;  Durham,  2006).  We  did  so, 
but  this  chapter  continues  to  draw  only  on  unclassified  materials  and 
deliberately  excludes  some  options  and  considerations  that  would  be 
examined  in  a  full  study  of  Global  Strike  options.37 

What  follows  is  a  streamlined  version  of  what  was  outlined  in 
Chapter  Three.  In  successive  sections  we  (1)  define  the  Global  Strike 
mission;  (2)  define  a  parameterized  scenario  space  and  a  small  span¬ 
ning  set;  (3)  define  potential  CONOPS  and  critical  components  of 
capability;  (4)  evaluate  effectiveness  of  the  baseline  and  options;  and 
(5)  assess  relative  cost-effectiveness.  We  also  include  a  section  on  the 
knotty  problem  of  how  to  discuss  risks  analytically  and  summarize 
results  compactly. 

Defining  Global  Strike 

Key  Attributes 

Ample  information  was  available  about  the  intent  of  the  Secretary  of 
Defense  and  the  President  on  the  matter  of  Global  Strike.  There  has 
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also  been  a  good  deal  of  public  discussion  and  review.38  Although  much 
of  the  public  discussion  is  concerned  with  nuclear  issues,  we  focused 
exclusively  on  conventional  strikes.  The  key  attributes,  or  defining  fea¬ 
tures,  are  as  follows: 

A  global  strike,  as  we  define  it,  is  a  discrete  strike,  not  an 
entire  campaign  or  even  a  bombing  campaign,  and  one  that 
is  explicitly  responsive  to  needs  and  concerns  of  the  Presi¬ 
dent,  Secretary  of  Defense,  and  other  top  leaders,  rather 
than,  say,  a  combatant  commander.39 

This  conception  is  very  similar  to  what  DoD  has  come  to  call 
Global  Strike  Raid,  a  subset  of  the  larger  Global  Strike.40 

A  broader  definition  of  Global  Strike  would  include  a  strike  to 
set  the  stage  and  “open  the  door”  for  a  large  conventional  campaign. 
It  would  also  include  a  series  of  small,  discrete  strikes  that  might  be 
used  to  disrupt  an  adversary’s  operations  and  buy  time  for  a  larger  U.S. 
response.  Such  extensions  would  imply  the  need  for  sizable  forces  and 
weapon  inventories.41 

Form.  Even  more  than  for  other  missions,  a  Global  Strike  force 
package  would  be  assembled  by  drawing  assets  from  all  major  com¬ 
mands  and  might  be  unique  to  the  strike  rather  than  something  more 
ordinarily  part  of  an  individual  combatant  commander’s  (COCOM’s) 
operations  plan.  The  Global  Strike  mission  should  be  conceived  broadly 
enough  so  that  execution  could  consist  of  or  include  Special  Operations 
Forces  (SOF).*  It  could  also  include  various  non-kinetic  mechanisms, 
such  as  information  operations  or  cyber  attacks. 

The  Executing  Command.  By  its  very  nature,  global-strike  capa¬ 
bility  cannot  reside  in  a  single  command.  Where  a  strike  would  be 
needed,  and  how  best  to  accomplish  it,  would  depend  on  the  situation. 
U.S.  STRATCOM  has  a  special  role  by  virtue  of  its  global  perspec¬ 
tive  and  control  of  strategic  forces,  but  at  the  time  of  a  strike  opera¬ 
tion,  command  might  be,  e.g.,  in  U.S.  CENTCOM  operating  in  the 


The  White  House  has  long-stated  interest  in  using  SOF  for  strikes  (Clarke,  2004,  pp.  141 — 
144).  The  2006  strike  into  Damadola,  a  remote  village  in  Pakistan,  was  reportedly  accom¬ 
plished  by  a  Predator  (Katzman,  2006,  p.  28.) 
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Middle  East  or  U.S.  PACOM  operating  in  East  Asia.  Such  complexity 
implies  the  need  for  unusual  peacetime  planning  and  coordination. 

Special  Considerations.  Global  Strike’s  execution  would  have  to 
be  unusually  sensitive  to  domestic  and  foreign-policy  considerations 
such  as  sovereignty  and  indirect  political  effects.  Further,  since  some 
of  the  circumstances  for  which  Global  Strike  capabilities  are  intended 
would  be  WMD-related  crises,  there  would  be  reason  to  worry  about 
collateral  damage42  and  about  issues  such  as  erroneous  perceptions  by 
the  target  or  other  countries  about  the  nature  of  attack,  conceivably 
triggering  escalation  or  use-it-or-lose-it  launches.  The  U.S.  Senate  has 
expressed  concerns  about  such  issues  (Woolf,  2007),  as  will  be  dis¬ 
cussed  later. 

Historical  Examples 

We  drew  on  historical  cases  to  sharpen  our  concept  of  Global  Strike. 
Analogs  to  global  strikes  in  the  past  include 

1 .  The  1980  failed  rescue  mission  in  Iran 

2.  The  1986  U.S.  strike  on  Libya 

3.  The  1981  Israeli  attack  on  the  Iraqi  nuclear  facility  at  Osirak 

4.  Various  strikes  against  Iraq  between  1995  and  2000  (e.g.,  the 
1998  strike  on  an  al  Qaeda  training  camp,  which  narrowly 
missed  Osama  bin  Laden) 

5.  A  strike  on  terrorist  camps  in  Afghanistan  (1998),  launched  in 
hopes  of  killing  bin  Laden,43  and  a  simultaneous  strike  of  a  sus¬ 
pected  chemical-weapons  facility  in  Sudan  (1998)44 

6.  The  bombing  of  Damadola,  a  village  deep  within  Pakistan, 
in  January  2006  in  an  attempt  to  kill  al  Qaeda’s  number-two 
figure,  Ayman  al  Zawahari.45 

Timeliness  Requirements 

A  global  strike  should  be  timely,  but  not  necessarily  “prompt,”  a  term 
interpreted  by  STRATCOM’s  General  Cartwright  and  others  to  mean 
within  about  an  hour  (Woolf,  2007,  p.  12).  In  our  view,  timeliness  does 
not  ordinarily  require  such  speed,  while  complex  decision  processes, 
checking  of  intelligence,  and  operational  planning  usually  are  needed.46 
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These  can  take  tens  of  hours,  or  even  days,  in  which  case  a  wide  range 
of  forces  could  be  used,  and  promptness  might  not  be  needed.  Strategic 
strikes,  even  conventional  strikes,  are  not  undertaken  lightly. 

This  said,  some  global  strikes  may  need  to  be  prompt,  i.e.,  within 
an  hour  or  so,  and  would  also  be  feasible.  The  key  factors  are  how 
long  the  target  will  be  vulnerable,  how  much  preparation  and  deci¬ 
sionmaking  are  necessary  at  the  time  (rather  than  earlier),  and  whether 
a  slower  attack  (such  as  one  with  aircraft  or  cruise  missiles)  might  be 
detected,  with  the  target  being  warned  and  able  to  escape.  We  dis¬ 
cuss  this  more  extensively  in  a  later  section,  along  with  various  ways  to 
improve  timeliness. 

The  United  States  currently  has  little  capability  to  accomplish  a 
prompt  global  strike.  Figure  5.1  illustrates  this  point  schematically, 
indicating  that  qualitatively  different  timescales  exist:  one  measured 
in  tens  of  minutes  up  to  about  an  hour,  one  measured  in  hours,  and 
one  measured  in  days.  Most  currently  plausible  Global  Strike  scenarios 

Figure  5.1 

A  Notional  Depiction  of  Current  Capabilities  for  Global  Strikes, 
by  Timescale 
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NOTE:  The  three  areas  denote  relative  frequency  of  need  for  very  short,  short,  and 
more-leisurely  execution  times.  The  perceived  need  for  fast  reactions  might  increase 
if  the  capability  existed  for  strikes  in  tens  of  minutes  or  an  hour.  The  figure  is 
intended  to  be  roughly  accurate  but  is  not  based  on  actual  calculations. 
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involve  many  hours  or  days  of  warning  and  preparation,  and  the  time 
to  actually  execute  a  mission  is  much  less  important  than  the  ability 
to  achieve  surprise  or  avoid  defenses.  The  United  States  already  has  a 
great  deal  of  capability  with  which  to  execute  strikes  of  this  type,  as 
indicated  by  the  green  region  in  Figure  5.1.  Some  Global  Strike  sce¬ 
narios  require  strikes  to  be  accomplished  within  hours,  which  is  pos¬ 
sible  using  forward-deployed  tactical  forces — if  they  are  in  the  right 
location  and  alerted,  //  they  can  be  employed  safely  without  extensive 
preparations  for  penetration  of  air  defenses,  and  //"their  forward  pos¬ 
turing  does  not  cause  trouble  in  itself  or  give  away  warning.47  We  show 
such  cases  in  yellow  in  Figure  5.1  because  what  is  feasible  depends  on 
details.  If  strikes  require  execution  times  of  only  an  hour  or  so,  the 
United  States  currently  has  no  non-nuclear  options  (except  perhaps  in 
cases  where  Tomahawk  platforms  happen  to  be  forward-deployed  close 
to  the  target  area).  It  follows,  as  suggested  by  Figure  5.1,  that  a  poten¬ 
tially  serious  shortfall  exists.  We  might  add  that  if  the  United  States 
had  prompt  strike  capability,  many  more  opportunities  needing  that 
capability  would  be  recognized. 

Using  these  definitions  and  distinctions,  we  now  move  on  to 
defining  a  parameterized  scenario  space. 


Defining  a  Parameterized  Scenario  Space  for  Global  Strike 

We  discussed  the  concept  of  a  spanning  set  of  scenarios  in  Chapter 
Three.  The  first  step  in  defining  one  is  to  conceive  the  broad  scenario 
space.  The  most-abstract  factors  that  should  go  into  defining  a  sce¬ 
nario  space  are  (1)  political-military  circumstances;  (2)  objectives, 
strategies,  and  tactics;  (3)  forces  and  weapons;  (4)  the  capabilities  of 
those  resources;  (5)  environmental  factors  (terrain,  weather,  etc.);  and 
(6)  other  model  and  data  assumptions  (Davis,  1994,  2002).  For  any 
given  application,  however,  we  need  more  concreteness  about  the  spe¬ 
cific  context. 

Our  context  was  defense  planning — i.e.,  developing  plans  and 
programs  for  future  military  capabilities.  Furthermore,  it  was  about 
“visible”  military  capabilities,  rather  than  magic  bullets  that  would — in 
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some  instances — be  able  to  exploit  special  vulnerabilities  of  a  particular 
adversary  known  from  sensitive  intelligence.  That  would  include  net¬ 
work  attacks  and  other  aspects  of  cyberwar.  The  pursuit  of  such  capa¬ 
bilities  is  best  discussed  elsewhere.48  We  also  excluded  activities  such 
as  attempting  to  “strike”  by  specifically  affecting  the  thinking  of  par¬ 
ticular  individuals  with  information  or  false  information,  or  by  physi¬ 
cally  apprehending  them.  Ultimately,  for  the  purposes  of  this  work,  we 
excluded  discussion  of  both  nuclear  and  non-kinetic  instruments  of  strike. 
We  focused  on  finding  a  scenario  space  dictated  largely  by  timeliness, 
target  character,  strike  size,  and  strategic  factors. 

Timeliness,  Target  Character,  and  Number  of  Weapons 

An  Abstracted  Design  Space.  A  first  task  is  characterizing  the 
scenario  space  and  its  distinguishable  regions.  Figure  5.2  does  this  for 
two  of  the  key  dimensions  (we  shall  discuss  others  later):  execution 
time  (x-axis),  i.e.,  the  time  between  the  order  to  execute  a  strike  and 
the  time  it  achieves  its  effect,  and  the  nature  of  the  target  (y-axis).  The 

Figure  5.2 

Regions  in  Design  Space 
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target-character  dimension  is  an  abstraction:  It  ranges  from  single  point 
targets  at  the  “easy”  end  (bottom);  to  buried  targets,  which  are  more 
difficult;  and  to  large,  deeply  buried,  and  ambiguous  targets,  which  are 
maximally  difficult  (top).*  By  “ambiguous”  we  mean  that  the  charac¬ 
ter,  size,  and  structure  of  the  underground  parts  of  the  target  are  not 
clearly  understood.  The  black  points  indicate  notional  scenarios  used 
to  represent  the  region  in  question  and  establish  reasonably  stressful 
“requirements.” 

Each  of  the  numbered  regions  indicates  the  boundary  of  appli¬ 
cability  for  a  given  system.  Thus,  tactical  aircraft  and  cruise  missiles 
are  associated  with  the  region  to  the  right  of  and  below  boundary  1 
(black).  Similarly,  boundaries  2,  3,  and  4  indicate  the  regions  applica¬ 
ble  to  bombers  (red),  ballistic  missiles  (brown  dashed),  and  joint  SOF 
options  (blue  dashed),  respectively^  The  solid  dots  indicate  representa¬ 
tive  points  to  which  test  scenarios  might  correspond.  That  is,  analysts 
need  not  consider  every  point  within  a  given  region,  but  only  such  a  set 
of  representative  points. 

Figure  5.2  suggests  that  tactical  aircraft,  cruise  missiles,  and 
bombers  can  cover  much  of  the  space,  but  ballistic  missiles  are  uniquely 
valuable  kinetic  mechanisms  for  times  of  less  than  a  few  hours.  Bomb¬ 
ers  have  special  capability  against  underground  targets;  even  bomb¬ 
ers,  however,  may  be  insufficient  against  the  more  ambiguous  FIDBTs, 
against  which  numerous  large  weapons  might  be  needed.  In  those  cases, 
we  postulated  that  the  facilities  might  have  special  weak  points,  which 
might  be  discovered  by  classic  intelligence  work,  cyberspace  methods, 
or  Special  Operations  Forces  (SOF).  For  the  sake  of  the  illustration,  we 
postulated  a  region  requiring  some  unspecified  joint  operation  involv¬ 
ing  a  combination  of,  e.g.,  bombers,  missiles,  and  SOF. 


DoD  has  long  described  hard  and  deeply  buried  targets  (HDBTs)  as  particularly  difficult 
challenges  (Department  of  Defense,  2000). 

^The  boundaries  are  fuzzy.  In  some  circumstances,  for  example,  aircraft  and  missiles  could 
have  shorter  execution  times  than  those  indicated  by  their  boundaries.  In  other  circum¬ 
stances,  times  would  be  longer  because  of  the  need,  e.g.,  for  a  carrier  battle  group  to  with¬ 
draw  from  another  operation  and  reposition  itself  for  a  strike  and  to  conduct  final  mission 
planning. 


54  Portfolio-Analysis  Methods  for  Assessing  Capability  Options 


Although  the  United  States  currently  has  no  capability  for  very 
short  execution  times,  it  could  obtain  that  capability  without  violating 
laws  of  physics  or  mining  Unobtanium.  Indeed,  the  Defense  Science 
Board  and  others  have  suggested  fitting  some  Trident  II  SLBMs  with 
conventional  warheads  (Defense  Science  Board,  2006),  and  Secretary 
Rumsfeld  decided  to  proceed  with  what  is  called  the  Conventional 
Trident  Modification  (CTM)  (Rumsfeld,  2006).  He  and  General 
Cartwright,  Combatant  Commander  for  U.S.  STRATCOM,  estab¬ 
lished  a  “requirement,”  one  that  is  not  arbitrary  or  a  “nice-to-have,” 
but  the  result  of  assessing  what  is  needed,  what  is  feasible,  and  at  what 
cost.  This  said,  Congress  has  not  yet  agreed  with  the  requirement  or 
the  solution,  and  the  issue  is  currently  being  debated.49  We  shall  return 
to  this  matter  later. 

Target  Character.  For  this  narrowed  but  still  broad  scope  of  Global 
Strike  Raid  missions,  it  is  possible  to  learn  much  about  the  appropriate 
scenario  space  by  reviewing  the  range  of  possible  targets.  We  developed 
a  taxonomy  of  targets,  as  shown  in  Table  5.1.  Target  characteristics 
are  discussed  in  technical  terms  such  as  “point”  and  “area,”  or  “hard” 
versus  “soft.”  The  rightmost  column  refers  cryptically  to  examples.  The 
taxonomy  of  target  types  is  rather  comprehensive50  and  distinguishes 
reasonably  well  among  the  many  target  circumstances  for  which  a 
global  strike  might  be  contemplated.  We  have  already  discussed  the 
timeliness  issue,  but  it  appears  in  the  taxonomy  as  well. 

As  discussed  in  a  later  section  on  execution,  the  requirement  to 
attack  these  many  classes  of  targets  has  important  implications  for  the 
mix  of  capabilities  needed. 

Attack  Size  (or  Weapon  Volume).  Another  dimension  that  mat¬ 
ters  in  defining  a  scenario  space  is  the  size  of  the  strike  needed.  As 
Figure  5.3  indicates,  we  envisioned  a  range  of  attacks  with  as  few  as  one 
weapon  to  perhaps  tens  of  weapons.  We  concluded  that  our  spanning 
set  should  include  at  least  one  case  each  for  the  low  end  and  the  high 
end  of  our  range. 

Strategic  Issues.  Of  the  many  strategic  issues  that  should  be  con¬ 
sidered  in  developing  a  scenario  space,  we  identified  bases  and  per¬ 
missions,  collateral  damage,  perceptions,  escalation  risk,  and  overall 
“plausibility.” 
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Table  5.1 

Illustrative  Taxonomy  of  Targets 

(not  including  non-physical  targets  or  targets  attacked  non-kinetically) 


Point  or 

Area?  Number? 

Time 

Sensitivity? 

Mobile? 

Buried? 

Hard? 

Examples 

Point  Small 

No 

No 

No 

No 

Higher-level  hq. 

Yes 

Yes 

WMD  facilities 

Yes 

No 

No 

Tactical  command 
posts 

Yes 

No 

No 

No 

Mobile-missile  sites 

Yes 

Nuclear  storage 
bunkers 

Yes 

Yes 

Leadership  sites 

Large 

No 

No 

No 

No 

Ports,  airfields, 
bases 

Partially 

Launch  complex 

Yes 

No 

No 

Exposed  aircraft, 
depots 

Yes 

No 

No 

No 

Ground  communi¬ 
cation  entry  sites 

Yes 

Missile  silos 

Yes 

No 

No 

Mobile  missiles 

Area  Small 

Yes 

NA 

No 

No 

Garrisons 

? 

Yes 

Yes 

Caves,  unidentified 
facilities 

Large 

? 

NA 

No 

No 

Urban  areas 

Yes 

NA 

Yes 

Yes 

Large  buried 
facilities 

A  Scenario  Set  That  Stresses  All  Key  Factors 

After  contemplating  the  various  issues  and  dimensions,  we  concluded 
that  a  reasonable  spanning  set  for  purposes  of  demonstrating  the  meth¬ 
odology — and  covering  the  principal  issues — would  need  three  classes 
of  scenario,  and  that  these  classes  could  be  represented  by  attacks 
on  (1)  mobile  missiles,  (2)  terrorist  leaders  in  an  urban  meeting,  and 
(3)  WMD  facilities.  Table  5.2  shows  how  each  of  the  factors  that  we 
believe  needed  to  be  stressed  would  be  stressed  by  suitable  scenarios 
in  the  three  classes.  The  point  of  Table  5.2  is  to  emphasize  that  the 
spanning  set  that  we  developed  was  the  result  not  merely  of  creative 
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Figure  5.3 

Possible  Weapon  Requirements  for  Strikes 
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scenario-spinning,  but  rather  of  an  analytical  process:  seeking  as  small 
a  set  of  scenarios  (and  cases  within  them)  as  needed  to  stress  develop¬ 
ments  in  all  of  the  appropriate  ways. 

In  our  illustration,  we  did  not  treat  several  important  dimensions 
that  we  most  certainly  would  have  treated  in  a  real  application.  These 
are  (1)  size  of  attack  needed,  (2)  target  mobility  at  the  time  the  strike 
reaches  the  target  area,*  and  (3)  the  value  of  the  various  options  as  the 
leading  edge  of  a  larger  attack. 

We  have  elaborated  on  the  spanning-set  concept  because  scenar¬ 
ios  are  often  conceived  poorly:  They  may  be  overly  stressful  in  some 
respects,  not  stressful  enough  in  others,  and  by  no  means  adequate — 
even  as  a  group — to  define  sound  design  requirements.  This  has  been 
recognized  for  some  years,  but  analytical  practice  has  tended  toward 
business  as  usual,  with  excessive  emphasis  on  working  through  details 
of  a  few  imperfectly  chosen  point  scenarios.51  In  any  case,  the  scenarios 
we  suggest  (black  points  in  Figure  5.2)  are  based  on  a  combination  of 
need,  technical  feasibility,  and  operational  feasibility. 


The  mobile  missiles  we  considered  move  from  their  peacetime  locations  to  operating  areas 
in  the  field  but  are  then  usually  in  definite  positions.  Actually,  moving  targets  are  easier  to 
deal  with  in  some  respects,  because  they  are  more  easily  distinguished  from  much  back¬ 
ground  clutter. 
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Table  5.2 

Degree  to  Which  Illustrative  Scenarios  Stress  Factors  of  Interest 


Factor 

Scenario 

Class  1 

(mobile  missiles) 

Scenario 

Class  2 

(terrorist  meeting) 

Scenario 

Class  3 

(WMD  facilities) 

Timeliness  and 
dependence  on  warning 

Bases  and  permissions 

•  • 

• 

•  •• 

Penetration  of  defenses 

•  ••• 

• 

•  •• 

Target  detection  and 

•  ••• 

• 

•  ••• 

characterization 

Target  destruction 

• 

• 

•  ••• 

Collateral  damage 

• 

•  • 

•  ••• 

Perceptions,  escalation 

•  • 

•  • 

•  ••• 

risk 

NOTE:  The  quantity  of  bullets  indicates  the  degree  to  which  the  factor  is  stressed.  The 
number  of  bullets  reflects  our  particular  analysis.  A  given  scenario  could  be  chosen  to 
be  more  stressful  in  other  ways.  For  example,  the  mobile-missile  and  terrorist-meeting 
scenarios  could  readily  stress  timeliness  and  collateral  damage,  respectively. 


Description  of  the  Scenario  Classes.  Table  5.3  provides  more 
detail  on  the  three  scenario  classes  of  our  spanning  set.  There  are  many 
instances  of  each  class;  for  example,  the  mobile  targets  might  be  trucks 
carrying  smuggled  WMD  or  related  materials,  and  the  hard,  fixed 
installations  might  be  command  posts.  Capability  against  one  of  our 
representative  scenario  classes  should  provide  capability  against  other 
members  of  that  class. 

The  first  scenario  class  is  represented  by  attacking  mobile  enemy 
missiles,  such  as  those  owned  by  either  a  rogue  or  a  peer.  The  mobile 
missiles  might  be  “generated”  (i.e.,  deployed  into  the  field  for  poten¬ 
tial  use)  and  might  even  be  nuclear-armed.  A  response  to  a  U.S.  strike 
must  be  considered  possible,  and  the  nature  of  that  response  might  be 
significantly  affected  by  the  success  of  the  strike  itself. 

The  second  scenario  class  would  have  fleeting  targets  to  be  killed. 
Such  a  scenario  might  have  high-ranking  terrorists  meeting  in  a  house 
for  some  hours,  with  the  United  States  knowing  in  advance  that  the 
meeting  will  take  place  in  a  particular  city,  but  not  knowing  until  late 
where  or  when  it  will  occur.  As  discussed  later,  a  real-world  strike  in 
1998  had  these  characteristics. 
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Table  5.3 

An  Approximate  Spanning  Set  of  Scenario  Classes  and  Cases 


Scenario  Class 

Key  Parameters  to  Test 

Contributing  Parameters 

1.  Attack  mobile 

Penetration  probability 

Air-defense  effectiveness 

weapon  systems 

Stealth  levels 

Penetration-aid  effectiveness 
levels 

Search  capability 

Search  area 

Search  rate 

Target-discrimination 

effectiveness 

Type  of  terrain 

Kill  capability 

Target  hardness,  location 
uncertainty,  weapon  type 

2.  Attack  terrorist 
leaders  in  urban 
setting 

Warning  and  decision  times 

Time  for  preparation  and  decision 

Required  execution  time 

Target  vulnerability  time 

Minimum  decision  time 

Maximum  collateral  damage 

Accuracy 

tolerable 

Fatalities  (or  other  effects) 

3.  Attack  WMD 

Penetration  probability 

Air-defense  effectiveness 

facilities 

Stealth  levels 

Penetration-aid  effectiveness 
levels 

Target  hardness 

Location  (and  distance  from 
bases) 

Target-location  uncertainty 

Degree  of  size  and  hardness  of 
target,  mapping  into  whether 
small  precision  bombs  are 
adequate 

Special  vulnerabilities 

Existence  of  known  exploitable 
special  vulnerabilities 

The  third  scenario  class  involves  attacking  a  nuclear-weapons  facil¬ 
ity  (or  a  facility  generating  or  storing  biological  or  chemical  weapons). 
Such  a  facility  might  be  underground,  might  not  be  well  characterized 
(in  which  case,  desired  aimpoints  would  be  unknown),  and  might  or 
might  not  be  well  defended.  An  attack  might  need  the  assistance  of  on- 
the-ground  intelligence  agents  or  SOF. 

Parameterization.  These  descriptions  are  quick  glosses.  The  next 
issue  is  parameterization.  The  primary  purpose  of  the  mobile-target 
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class  is  to  stress  C4ISR  and  end-game  weapon  agility.  The  degree  of 
stress,  however,  depends  on  a  number  of  factors.  Similarly,  the  point  of 
the  terrorist-meeting  scenario  is  to  stress  capabilities  for  prompt  preci¬ 
sion  attacks  with  minimum  collateral  damage.  But  how  much  warn¬ 
ing  would  be  given,  and  how  long  would  the  meeting  last?  How  much 
uncertainty  would  there  be?  As  for  WMD  facilities,  some  are  above 
ground,  some  are  below  ground  but  well  located,  and  still  others  are 
below  ground  and  not  well  located.  Imperfect  targeting  could  cause 
the  release  of  toxic  materials,  resulting  in  extensive  collateral  damage, 
extremely  damaging  international  perceptions,  or  both.  All  such  issues 
can  be  parameterized  for  exploratory  analysis  by  varying  assumptions. 

The  rightmost  column  of  Table  5.3  shows  the  parameters  we 
used  to  explore  some  of  the  questions.  If  we  elaborated  the  table,  we 
could  assign  values  (e.g.,  low,  medium,  and  high)  to  parameters.  In  our 
analysis,  these  were  mapped  into  more-precise  expressions.  For  exam¬ 
ple,  an  advanced  air  defense  corresponded  to  “double-digit”  surface- 
to-air  missiles  (SAMs)  of  the  sort  produced  by  Russia  and  available  on 
the  world  market.  Manned  bombers  can  be  characterized  as  having 
high,  medium,  or  low  levels  of  stealth,  which  can  then  be  assumed  to 
have  high,  medium,  or  low  ability  to  penetrate  the  various  classes  of 
air  defense.  Going  beyond  this  type  of  purely  parametric  work  would 
require  classified  levels  of  analysis. 

One  other  point  should  be  made  about  the  contributing  param¬ 
eters  in  Table  5.3.  We  defined  them  at  different  levels  of  resolution  so 
that  we  could  input  something  like  a  penetration  probability  directly 
or,  with  more  effort  and  complexity,  we  could  enter  inputs  for  the  air- 
defense  level,  the  level  of  stealth,  and  the  level  of  penaid  effectiveness. 
That  multiresolution  approach  is  quite  useful  in  practical  analytical 
work.52 

More  on  the  Timeline  Issue.  Table  5.3  indicates  that  timeline 
parameters  are  important  for  scenario  class  2.  Figure  5.4  illustrates 
schematically  the  timeline  issue  for  a  nominal  case  in  which  the  order 
of  events  is  strategic  warning,  actionable  warning  and  the  beginning 
of  top-level  decisionmaking,  alerting  of  forces,  the  order  to  execute, 
and,  finally,  the  time  that  effects  occur  (e.g.,  a  target  is  destroyed). 
Strategic  warning  might  occur  months  in  advance  (the  United  States 
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Figure  5.4 

An  Illustrative  "Normal"  Timeline 
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already  has  strategic  warning  about  the  need  to  attack  al  Qaeda  leaders, 
for  example),  whereas  actionable  warning  might  come  within  a  day  or 
even  hours  of  when  a  strike  will  be  needed.  The  order  to  execute  might 
come  a  day  or  so  in  advance,  or  perhaps  only  an  hour  or  so  before  the 
effects  of  a  strike  are  needed  on  the  target.  In  one  historical  example, 
the  U.S.  strike  on  al  Qaeda  training  camps  in  1998  was  motivated  by 
intelligence  that  bin  Laden  would  appear  at  one  of  the  camps  on  a  spe¬ 
cific  day  a  few  days  hence.  Deliberations  began  and  forces  were  alerted. 
When  the  order  to  execute  was  given,  flight  time  for  the  cruise  missiles 
was  about  two  hours — after  days  of  preparation  (Clarke,  2004;  Clin¬ 
ton,  2004).  According  to  some  accounts,  the  strikes  were  just  a  bit  too 
slow. 

The  case  illustrated  in  Figure  5.4  is  only  one  of  a  number  of  pos¬ 
sibilities,  because  even  the  order  of  events  in  the  timelines  is  uncertain. 
Figure  5.5  shows  one  contrasting  possibility.  In  this  case,  because  of 
prior  intelligence  and  decisionmaking,  there  is  no  gap  between  action¬ 
able  warning,  alerting,  and  the  ordering  of  execution.  Execution  might 
be  ordered  immediately  upon  receiving  actionable  warning  consistent 
with  the  strict  criteria  of  a  decision.  Further,  in  this  case,  the  execution 
time  would  be  the  limiting  factor  in  determining  the  relative  goodness 
of  Global  Strike  options. 
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Figure  5.5 

An  Illustrative  Timeline  with  Prior  Preparations  and  Decision 
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Yet  another  possibility  is  that  a  local  commander  could  act  imme¬ 
diately  upon  warning  because  of  having  been  delegated  to  do  so.  In 
practice,  it  is  very  likely  that  U.S.  strategic  strikes,  even  if  of  a  very  lim¬ 
ited  nature,  will  be  strongly  and  centrally  controlled.  Some  opportuni¬ 
ties  may  be  lost,  but  the  implications  of  actually  conducting  a  strike  are 
so  great  that  extreme  precautions  are  merited. 

Within  both  Figures  5.4  and  5.5,  the  parameter  of  greatest  impor¬ 
tance  analytically  is  the  latest  of  Tw  and  Te,  the  times  of  actionable 
warning  and  preparation,  and  decision  and  execution,  respectively.53 

Characterizing  Comprehensiveness.  The  scope  of  our  approxi¬ 
mate  spanning  space  is  conveyed  compactly  in  the  “spider  chart”  shown 
in  Figure  5. 6. 54  It  could  be  briefed  at  a  review  meeting  as  follows  (if 
participants  already  understood  spider  charts): 

Before  showing  results  of  our  comparison  of  options,  remem¬ 
ber  that  we  have  not  considered  all  possible  uses  of  Global 
Strike  for  various  reasons  relating  to  feasibility,  plausibility, 
and  the  time  available  to  us.  We  have  sought  to  be  quite 
demanding  in  looking  at  the  timeline  issue  (including  depen¬ 
dence  on  warning),  degrees  of  vulnerability  of  the  targets  to 
our  weapons  systems,  C4ISR  issues  related  to  detection  and 
tracking,  and  the  potential  for  a  reactive  threat.  We  have 
been  less  comprehensive  in  thinking  through  the  different 
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Figure  5.6 

Coverage  in  Assumptions  Space 


Timeliness 


NOTE:  The  contour  shows  schematically  the  degree  to  which  analysis  stresses  the 
various  factors:  High  numbers  (toward  the  outside)  correspond  to  high  stress.  Our 
analysis  stresses  timeliness  and  target  vulnerability  heavily  and  C4ISR  and  political- 
military  considerations  significantly;  it  excludes  cyberwar  and  other  non-kinetic 
options. 
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political  and  operational  constraints  that  might  apply.  And, 
because  we  looked  at  small  strikes  for  very  limited  purposes, 
our  analysis  did  not  address  the  size  of  attack  needed,  for 
example,  as  the  leading  edge  of  a  large  strike.  Also,  in  this 
analysis,  we  do  not  treat  anything  related  to  cyberwar,  such 
as  network  attacks,  or  to  other  non-kinetic  mechanisms  of 
attack.  Those  will  be  addressed  separately. 

Having  developed  our  approximate  spanning  set  of  parameter¬ 
ized  scenarios,  let  us  now  move  on  to  concepts  of  operations  and  criti¬ 
cal  components. 
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Defining  CONOPS  and  Critical  Components 
for  Global  Strike 

Critical  Components  of  Capability 

Successful  global  strikes  could  depend  on  many  factors,  but  the  cat¬ 
echism  of  our  approach  includes  boiling  these  down  and  develop¬ 
ing  hierarchies.  From  a  high-level  perspective,  we  identified  three 
aggregate-level  critical  components:  (1)  military-technical  effectiveness, 
(2)  political-military  effectiveness,  and  (3)  price  of  executing  the  mis¬ 
sion  (loss  of  personnel  or  equipment).  For  each,  it  is  necessary  to  distin¬ 
guish  between  “expected”  results  and  what  might  happen,  to  appropri¬ 
ately  address  risks.  We  address  the  second  and  third  components  first 
and  then  discuss  military-technical  effectiveness  in  more  detail. 

Political-Military  Effectiveness.  Ultimately,  a  global  strike  may 
be  intended  to  accomplish  more  than  destroying  one  or  a  few  targets. 
The  likely  intended  effect  may  be  more  political-military  than  military- 
technical.  It  may  sometimes  be  social.  In  addition,  various  effects  will 
be  very  undesirable.  Will  the  collateral  damage  be  acceptable,  given  the 
circumstances?  Will  execution  of  the  strike  be  tolerable  in  terms  of,  say, 
respect  of  sovereignty,  coordination  with  allies,  or  even  integration  of 
allies  in  the  operation?  Will  perceptions  about  the  strike — locally  and 
globally — be  as  desired?55  Do  the  various  options  for  Global  Strike  vary 
in  their  likely  effects  or  related  risks?  If  so,  that  information  should  be 
included  in  the  assessment. 

Price  of  Executing  the  Mission.  What  would  be  the  likely  or  pos¬ 
sible  price  in  terms  of,  e.g.,  the  loss  of  pilots,  SOF  personnel,  and  mili¬ 
tary  equipment?  This  question  might  not  seem  very  strategic  at  first 
blush,  but  the  American  military  ethos  calls  for  extraordinary  efforts  to 
protect  and  recover  personnel.  Moreover,  there  are  dangers  associated 
with  captures,  such  as  show  trials,  loss  of  leverage,  and  loss  of  sensitive 
equipment.  Thus,  such  considerations  should  be  evaluating  options  for 
a  global  strike. 

Military-Technical  Effectiveness.  The  primary  measure,  of  course, 
is  military-technical  effectiveness,  or  what  we  shall  call  “execution 
effectiveness.”  We  deal  with  that  in  the  next  section  in  connection  with 
concepts  of  operations. 
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Concepts  of  Operations 

Options  cannot  be  evaluated  for  effectiveness  without  specifying  con¬ 
cepts  of  operations  (CONOPS).  Not  much  detail  is  necessary  for  our 
purposes.  We  assumed,  for  each  CONOPS  used,  that  the  United 
States  would  have  adequate  strategic  warning  and  long  lead-time  prep¬ 
aration.  We  also  assumed  effective  command  and  control.  For  Global 
Strike,  this  could  mean  substantial  dependence  on  the  Global  Infor¬ 
mation  Grid,  with  information  flow  among  airborne,  space-based,  and 
ground-based  sensors,  command-and-control  centers,  supporting  orga¬ 
nizations,  and  tactical  operations. 

Given  these  circumstances,  any  CONOPS  for  our  various  options 
must  deal  with 

1.  Penetration  of  weapons  to  the  target  (and  egress,  for  manned 
aircraft) 

2.  Timeliness  of  the  strike 

3.  Finding  the  target  (i.e.,  some  combination  of  detecting,  locating, 
characterizing,  or  tracking) 

4.  Destroying  the  target  (or  functionally  disabling  it). 

As  examples,  manned  bombers  need  to  penetrate  air  defenses, 
find  the  target,  destroy  it,  and  return;  ballistic  missiles  do  not  usually 
have  to  deal  with  defenses  and  would  not  need  to  return,  but  still  need 
to  find  and  destroy  the  target;  SOF  units  would  need  to  penetrate  in¬ 
country  defenses  covertly  (e.g.,  by  coastal  patrols,  border  guards,  infan¬ 
try  protecting  installations,  and  aircraft  responding  to  alerts),  conduct 
their  mission  (e.g.,  target-spotting  or  characterization,  or  even  direct 
attack),  and  escape. 

In  what  we  call  a  “joint  CONOPS”  (a  bit  of  a  misnomer  because 
all  of  the  concepts  of  operations  would  be  joint  in  one  way  or  other), 
combination  actions  would  be  used.  For  example,  aircraft  and/or  mis¬ 
siles  could  attack  defenses,  while  SOF  units  could  then  attack  the 
target;  or  SOF  units  could  help  with  detection  and  tracking  but  leave 
target-killing  to  aircraft  and/or  missiles.  In  some  cases,  targeting  might 
be  provided  by  intelligence  assets  or  friendly  foreign  nationals.  All  of 
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these  have  precedents  in  the  1991  Iraq  war  and  U.S.  conflicts  in  Iraq 
and  Afghanistan. 

Each  of  the  above  items  is  a  critical  factor,  as  are  the  political- 
military  effects  and  the  price  of  execution  mentioned  earlier.  It  follows 
that  our  analytic  framework  and  methods  must  deal  with  all  of  them. 


Identifying,  Evaluating,  and  Comparing  Global  Strike 
Options 

The  Options  Themselves 

We  began  with  a  set  of  building  blocks  (described  below),  experimented 
using  our  option-generation-and-screening  tool,  BCOT,  as  described 
in  Chapter  Four,  and  iterated.  The  result  was  a  shorter  set  described  in 
the  second  subsection  below. 

Initial  Building  Blocks.  The  baseline  program  is  the  natural  first 
option  to  examine.  It  includes  substantial  capability  in  the  form  of 
B-2Bs,  Air  Force  and  Navy  tactical  aircraft,  conventional  cruise  mis¬ 
siles,  Special  Operations  Forces  (SOF),  precision  weapons  of  various 
types,  satellite  communications,  national  intelligence  systems,  regional 
C4ISR  systems  such  as  J-STARS,  and  so  on.  If  the  purpose  of  the 
analysis  is  to  find  tradeoffs  and  bill-payers,  the  baseline  might  be  the 
current  program  of  record  minus  elements  to  be  computed  again,  but 
in  our  application,  there  were  no  obvious  elements  of  this  type  within 
the  Global  Strike  Raid  domain. 

The  building-block  options  add  capability  to  the  baseline.  After 
early  discussions,  we  decided  to  use  the  building-block  options  shown 
in  Table  5.4.  Many  of  these  involve  systems  with  multiple  purposes, 
but  here  we  consider  only  their  value  for  Global  Strike  missions. 

If  expensive  systems  are  procured  specifically  for  Global  Strike, 
we  should  also  include  “negative  building  blocks,”  i.e.,  building  blocks 
calling  for  reductions  of  the  baseline  program  in  one  way  or  another. 
That  was  not  relevant  for  the  Global  Strike  Raid  problem,  because  the 
baseline  program  does  not  include  sizable  chunks  specifically  for  Global 
Strike  Raid.  Instead,  bill-payers  must  ultimately  come  from  capabilities 
usually  associated  with  other  missions.56 
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Table  5.4 

Building-Block  Options 


Building  Blocks  and  Abbreviations  Motivation  in  Global  Strike  Context 


Forward  basing  of  B-2s  (forw.  basing) 

Penetration  aids  (penaids)  for  Air  Force 
tactical  aircraft 

Penetration  aids  (penaids)  for  Navy 
tactical  aircraft 

Space-based  synthetic-aperture  radar 
(SAR)  and  moving-target-indicator 
(MTI)  radar 

Automated  target-recognition  systems 
(ATRs) 

Enhanced  insertion  vehicle  for  SOF 
(SOF  vehicle) 

Armed  Predators  for  SOF  (Predators) 

Air  Force  conventional  ICBM  (ICBM) 

Navy  submarine-launched  conventional 
ballistic  missile  (SLBM),  such  as  the 
Conventional  Trident  Modification 
(CTM) 

Advanced  manned  bomber  (adv. 
bomber) 

Joint  common  aerospace  vehicle  (JCAV) 


To  reduce  timelines 

To  increase  penetration  capability  against 
advanced  SAMs 

To  increase  penetration  capability  against 
advanced  SAMs 

To  enhance  detection  and  tracking  of 
mobile  targets,  among  other  capabilities 

To  enhance  detection  and  identification 
of  targets  by  distinguishing  them  from 
ground  clutter 

To  enhance  deep,  covert  penetration  into 
defended  areas 

Additional  armed  Predators  specifically  to 
allow  SOF  units  to  stand  off  from  targets 

To  provide  a  conventional  option  for 
strategic  forces 

To  provide  a  conventional  option  for 
strategic  forces 


To  increase  penetration  of  even  the  most 
advanced  SAMs  and  to  avoid  discovery 

To  develop  improved  search,  track,  and  kill 
for  ballistic  missiles  and  to  operate  on 
both  Air  Force  and  Navy  missiles 


NOTE:  Building  blocks  are  additive  to  the  baseline  program,  which  includes  B-2s, 
tactical  aircraft,  cruise  missiles,  SOF,  and  C4ISR  systems.  Items  in  parentheses  are  short 
names  used  in  subsequent  tables  and  figures. 

Our  initial  building-block  options,  listed  in  Table  5.4,  were  moti¬ 
vated  by  concerns  raised  in  connection  with  the  scenario  set  described 
in  the  previous  section.  They  involved  ability  to  penetrate  advanced 
air  defenses  (“double-digit  SAMs,”  potentially  networked);  ability  to 
detect,  find,  and  track  mobile  targets;  timeliness;  ability  to  deliver  the 
payloads  needed  for  attack  of  some  targets;  and  ability  of  SOF  forces 
to  penetrate. 

Our  list  includes  nothing  exotic.  Forward  basing  of  bombers 
would  be  a  logical  (even  if  sometimes  impractical)  way  to  shorten  time- 
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lines  for  bombers.  Penetration  aids  for  tactical  aircraft  have  long  been 
planned  as  part  of  a  natural  evolution. 

Space-based  synthetic-aperture-radar  (SAR)  systems  for  com¬ 
bined  defense  and  intelligence-community  applications  have  been  in 
development  within  the  Space  Radar  program.  According  to  the  pro¬ 
gram  manager  in  2005  (Sheridan,  2005): 

The  main  capabilities  we’re  talking  about  for  Space  Radar  are 
to  provide  a  day-night,  all-weather,  synthetic-aperture  radar; 
surface  moving-target  indication  collection  capability;  high- 
resolution,  terrain-imaging  capability;  advanced-geospatial  intel¬ 
ligence  capability;  and  an  open-ocean  surveillance  capability.  We 
plan  to  be  able  to  gather  and  collect  information  in  all  those  types 
of  formats  and  to  process  it  and  to  provide  it  back  to  users  as 
needed. 

Automated-target-recognition  (ATR)  systems  have  been  in 
research  and  development  (R&D)  for  decades,  and  considerable  prog¬ 
ress  has  been  made,  albeit  with  continuing  challenges  and  the  need  for 
dramatic  increases  in  sensor  capability.  MIT’s  Lincoln  Laboratory  has 
been  a  leader  in  such  work  (Vasile  and  Marino,  2005). 

The  SOF  insertion  vehicle  to  which  we  refer  is  notional,  but  U.S. 
SOCOM  is  always  looking  into  such  advanced  capabilities,  which  are 
offered  up  by  industry  in  a  variety  of  forms,  are  in  different  states  of 
R&D,  and  involve  different  degrees  of  technological  risk.  In  our  analy¬ 
sis,  the  capabilities  are  merely  treated  as  parameters  with  a  range  of 
possible  values.  Stealthy  SOF  penetration  is  achieved,  of  course,  by  a 
combination  of  factors,  including  the  launch  platform,  the  detectabil¬ 
ity  of  the  insertion  vehicle,  and  the  tactics  of  choosing  flight  paths. 

Armed  Predators  for  SOF  are  an  obvious  possibility,  given  the 
general  procurement  of  Predators  and  past  examples  of  the  use  of  Pred¬ 
ators  with  weapons  to  attack  Middle  Eastern  terrorist  targets  directly. 

The  Air  Force  Minotaur  program  is  looking  at  a  conventional 
ICBM  option  based  significantly  on  preexisting  missile  components. 
This  conceptual  missile  was  compared  with  the  nearer-term  option 
for  the  Navy’s  Conventional  Trident  Modification  (CTM)  in  a  recent 
news  article  based  on  an  interview  with  Air  Force  Col.  Rick  Patenaude 
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(Strak,  2006).  A  more  extensive  discussion  has  been  published  by  the 
Congressional  Research  Service  (Woolf,  2007).  In  addition,  the  U.S. 
Senate  asked  a  National  Academy  panel  to  review  some  of  the  issues 
and  options  relating  to  prompt  nuclear  strike.  The  academy’s  interim 
letter  report  recommended  going  ahead  with  full  R&D  for  the  CTM 
(including  testing)  but  recommended  another  year’s  delay  before  decid¬ 
ing  on  acquisition  and  fielding.  It  also  recommended  pursuing  R&D 
on  other  missile  options  such  as  boost-glide  systems,  including  the 
Minotaur  and  an  Army  system,  but  described  them  as  being  longer- 
term  options  with  more  technical  risk  (National  Research  Council, 
2007). 

One  building  block  that  we  might  have  included  is  the  sea- 
launched  global-strike  missile  (SLGSM),  which  could  be  deployed  on 
submarines,  surface  ships,  or  both.  It  is  currently  seen  as  a  potential 
follow-on  to  the  CTM  and  would  benefit  substantially  from  experience 
with  the  CTM. 

As  for  advanced  bombers,  a  next-generation  long-range  strike 
(NGLRS)  platform  has  been  under  consideration  with  industry  devel¬ 
oping  manned  and  unmanned  alternative  concepts  for  the  Air  Force 
and  even  a  white  paper  (U.S.  Air  Force,  2001).  Chief  of  Staff  General 
Moseley  and  Air  Force  Secretary  Wynne  have  discussed  the  NGLRS 
concept  in  interviews  and  in  Congressional  testimony.  The  NGLRS 
concept’s  purpose  seems  much  broader  than  Global  Strike  Raid. 

The  JCAV  that  we  postulated  does  not  seem  to  exist  as  yet.  Instead, 
it  appears  that  the  Navy  and  Air  Force  ICBM  programs  are  developing 
separate  front  ends,  at  least  in  part  because  the  missile  diameters  are 
different  and  because  of  additional  design  constraints  for  systems  to  be 
operated  in  a  submarine. 

An  Iterated  Set  of  Building  Blocks.  Based  on  extensive  work  with 
our  screening  tool,  BCOT,  we  considered  thousands  of  composite 
options  (options  to  buy  various  combinations  of  one  or  more  build¬ 
ing  blocks).  Our  intention  was  to  take  a  broad  view  initially  and  then 
screen.  We  shall  spare  the  reader  the  description  of  our  discussions 
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along  the  way  and  merely  summarize  our  results,  which  led  to  an  itera¬ 
tion  of  building  blocks.  The  conclusions  were  the  following: 

•  The  armed  Predator  (or  a  functional  equivalent)  should  be  consid¬ 
ered  on  its  broad  merits  and,  if  found  attractive,  included  in  all  of 
the  composite  options  considered  further.  One  reason  for  this  is 
that  the  option  has  a  relatively  low  price  tag. 

•  If  penetration  aids  are  needed  for  current-generation  fighter  and 
bomber  aircraft,  they  should  logically  be  procured  for  both  Air 
Force  and  Navy  systems.  Thus,  we  introduced  a  new  building 
block  called  “joint  penetration  aids.”  In  practice,  they  would 
have  Service-specific  differences,  but  those  are  matters  for  higher- 
resolution  analysis. 

•  It  would  probably  make  sense — from  a  strictly  Global  Strike 
perspective — to  procure  a  satellite  constellation  with  synthetic- 
aperture  radars  (SARs)  and  capabilities  for  automated  target  rec¬ 
ognition  (ATR)  on  board  penetrating  aircraft.  In  operational 
analysis,  both  would  probably  be  necessary  in  the  scenarios  for 
which  either  would  potentially  be  most  useful.*  Thus,  we  sub¬ 
stituted  a  combined  option  for  both  ATR  and  SARs,  calling  it 
“sensors”  for  short. 

•  Our  notional  analysis  was  to  be  indifferent  to  whether  the 
advanced  bomber  should  be  a  very  stealthy,  high-capacity,  long- 
range  bomber  available  perhaps  by  2025,  or  a  medium-range  air¬ 
craft,  perhaps  available  as  early  as  2018  as  mandated  by  the  Qua¬ 
drennial  Defense  Review  for  less  than  compelling  reasons.  The 
cost  of  such  platforms  would  differ  significantly,  of  course,  but  we 
were  not  attempting  to  use  accurate  cost  data  in  any  case. 

•  We  reluctantly  dropped  the  concept  of  the  JCAV  because  it 
appeared  highly  speculative  and  unsupported,  and  retaining 
the  option  would  have  added  complexity  of  little  interest  for  the 
notional  analysis. 


This  conclusion  is  arguable,  since  ATRs  are  merely  an  efficient  mechanism  for  identifying 
targets  amid  clutter,  and  such  data  analysis  need  not  necessarily  be  accomplished  on-board 
a  penetrating  aircraft. 
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Table  5.5  summarizes  the  iterated  set  of  building  blocks  used  in 
the  next  phase  of  our  analysis. 

Finding  Candidate  Options  for  a  Fuller  Analysis 

Analysis  Based  on  Nominal  Parameter  Values.  We  continued  to 
exercise  BCOT,  considering  thousands  of  possible  combinations  of  the 
building  blocks  in  Table  5.5.  We  sought  a  screen  for  a  small  set  of  com¬ 
posite  options  that  would  make  sense  from  an  effectiveness  perspective. 
For  a  given  set  of  assumptions — akin  to  analysis  based  on  a  DoD- 
specified  set  of  scenarios  and  a  DoD-specified  database  of  scenario 
data,  including  assumptions — we  used  BCOT  to  generate  diagrams 
like  Figure  5.7,  which  shows  a  simplified  measure  of  effectiveness  for 
each  composite  option,  i.e.,  each  point  in  the  figure.  The  dominant 
points  are  those  on  the  efficient  frontier,  which  is  the  solid  line.  At 
a  given  program  cost,  the  most  effective  option  available  is  no  more 
effective  than  the  value  shown  on  the  efficient  frontier,  and  the  “best” 
option  will  be  the  point  on  that  frontier  at  or  below  the  cost  level  in 
question.  The  dominant  points,  then,  are  the  best  points  identified  in 
a  first  look. 

Table  5.5 

Iterated  Set  of  Building  Blocks 


Building  Block  Motivation  in  Global  Strike  Context 

Forward  basing  of  B-2s  To  reduce  timelines 

Penetration  aids  (penaids)  for  Air  Force  and  To  increase  penetration  capability 
Navy  aircraft  against  advanced  SAMs 

Space-based  synthetic-aperture-radar  To  enhance  detection  and  tracking  of 
(SAR)  and  automated-target-recognition  mobile  targets;  other, 

systems  (ATR)  for  Air  Force  and  Navy 
aircraft 

Enhanced  special  insertion  vehicle  (SIV)  for  To  enhance  deep,  covert  penetration  of 
SOF  defenses 

Air  Force  conventional  ICBM  To  provide  a  conventional  option  for 

strategic  forces 

Conventional  Trident  Modification  (CTM)  To  provide  a  conventional  option  for 

strategic  forces 

Advanced  manned  bomber  To  increase  long-range  penetration  of 

even  the  most  advanced  SAMs 
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Figure  5.7 

Finding  the  Dominant  Options 


Apportioned  costs  (Sbillions) 

NOTE:  Each  point  is  a  composite  option.  The  plot  was  generated  before  we  had 
narrowed  the  range  of  building  blocks  deemed  to  be  useful.  As  a  result,  it  contains 
thousands  of  points. 
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Exploratory  Analysis  Across  Parameter  Uncertainty.  As  dis¬ 
cussed  in  a  companion  report  on  BCOT  (Davis,  Shaver,  Gvineria,  and 
Beck,  2007),  a  number  of  considerations  temper  the  neatness  of  this 
mathematical  analysis.  First,  we  want  to  consider  not  only  dominant- 
point  options,  but  also  some  that  are  near  the  efficient  frontier.  Some 
of  those  may  prove  superior  under  more- extensive  analysis  with  addi¬ 
tional  measures  of  effectiveness  and  risk.  Second,  the  determination 
of  which  options  are  dominant  (or  near  the  efficient  frontier)  depends 
sensitively  on  many  assumptions  about  the  warfighting  scenarios,  com¬ 
ponent  capabilities,  force  employment,  costs,  and  so  on.  In  particular, 
the  dominance  of  the  options  depends  on  the  relative  emphasis  placed 
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on  the  different  scenarios  (something  we  call  “focus”)  and  the  different 
cases  within  them.* 

Table  5.6  illustrates  how  the  identity  of  the  dominant  points — 
the  strongest  candidates  for  a  full  portfolio  analysis — varies  depending 
on  which  scenario  classes  are  considered  and  how  their  relative  impor¬ 
tance  is  weighted. 

As  expected,  options  involving  advanced  sensors  (the  SAR  and 
ATR  building  blocks)  show  up  only  if  the  mobile-missile  scenario  is 
either  emphasized  or  at  least  given  weight  equal  to  that  of  the  other 
scenarios  (the  first  and  fourth  columns,  respectively).  Which  options 
appear  as  dominant  points  also  depends  on  many  parameter  assump¬ 
tions,  so  Table  5.6  is  merely  one  of  many  possible  displays.  Still,  it  dra¬ 
matizes  the  point  that  the  attractiveness  of  options  depends  strongly  on 
the  criteria  used. 

After  studying  such  results,  we  concluded  the  following: 

•  The  options  for  more-extensive  analysis  should  be  the  combina¬ 
tions  of  those  that  do  well  for  the  different  “focuses,”  i.e.,  options 
that  are  dominant  for  the  mobile-missile  scenario  class  or  the  ter¬ 
rorist-scenario  class  or  the  WMD-facility  scenario  class. 

•  Similarly,  we  should  retain  an  option  even  if  it  is  dominant  only 
for  special  (but  important)  cases  within  a  single  scenario  class. 

•  Finally,  we  should  retain  some  options  that  are  near  the  efficient 
frontier  but  not  on  it  if  they  have  special  features  making  them 
potentially  attractive  in  a  fuller  analysis. 

•  Iteration  is  necessary:  After  comparing  options  in  the  full  portfo¬ 
lio  structure  (next  section),  we  may  find  some  options  that  were 
initially  screened  out  to  be  more  attractive.  Redoing  the  BCOT 
analysis  with  a  different  effectiveness  function  would  then  be 
desirable. 


Numerous  options  may  have  effectiveness  as  good  as  that  of  a  dominant  point  and  be  only 
slightly  more  costly  (see  Figure  5.7).  Thus,  there  may  be  too  many  options  near  the  efficient 
frontier.  BCOT  includes  an  algorithm  that  uses  set  theory  to  eliminate  the  uninteresting 
points. 
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Table  5.6 

Sensitivity  of  the  List  of  Candidate  Options  to  Focus 


Scenario 

Scenario 

Scenario 

Average  over 

Class  1: 

Class  2: 

Class  3: 

Average  over 

Terrorist  and 

Mobile  Missiles 

Terrorists 

WMD  Facilities 

All  Scenarios 

WMD  Scenarios 

Sensors 

Forw.  basing 

Penaids 

Forw.  basing 

Forw.  basing 

Penaids  + 

SLBM 

Adv.  bomber 

Penaids 

Penaids 

sensors 

Sensors  +  adv. 

Forw.  basing  + 

Penaids  +  SOF 

Forw.  basing  + 

Forw.  basing  + 

bomber 

Penaids 

vehicle 

penaids 

penaids 

Adv.  bomber  + 

Forw.  basing  + 

Forw.  basing  + 

SOF  vehicle 

adv.  bomber 

adv.  bomber 

Forw.  basing  + 

Forw.  basing  + 

SOF  vehicle  + 

SOF  vehicle  + 

penaids 

penaids 

Forw.  basing 

Forw.  basing  + 

+  sensors  + 

sensors  +  adv. 

penaids 

Sensors  +  SOF 
vehicle  +  adv. 
bomber 

Adv.  bomber  + 
sensors  +  SOF 
vehicle  +  adv. 
bomber 

Sensors  +  SOF 
vehicle  +  adv. 
bomber 

bomber 

Forw.  basing  + 
sensors  +  SOF 
vehicle  +  adv. 
bomber 

NOTE:  Options  in  each  column  are  on  or  near  the  efficient  frontier  for  the  "focus" 
indicated  by  the  column  title.  Different  focuses  imply  different  relative  weighting 
of  scenarios  in  calculating  effectiveness.  The  first  three  columns  consider  only  one 
scenario;  the  "all  scenarios"  column  weighs  the  three  scenarios  equally;  the  rightmost 
column  averages  over  the  second  and  third  scenario  classes. 


Mathematically,  the  first  conclusion  corresponds  to  using  the 
union  of  option  sets  that  are  attractive  in  one  or  another  interesting 
class  of  assumptions.  The  last  conclusion  means,  in  practice,  that  we 
examined  options  near  the  efficient  frontier  one  by  one  before  decid¬ 
ing  whether  to  cull  them.  Thus,  the  methodology  is  computer- assisted, 
not  fully  automated.  This  is  a  strength,  not  a  weakness,  because  ana- 
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lysts  will  almost  always  have  knowledge  not  represented  in  the  rela¬ 
tively  simple  effectiveness  function  used  in  an  option-generating-and- 
screening  tool.  We  have  no  particular  interest  in  a  clever  automated 
tool  that  precludes  the  easy  application  of  common  sense.  Besides,  the 
time  required  for  an  analyst  to  read  through  a  near-final  list  (perhaps  a 
day?)  is  small  compared  with  the  time  required  for  analysis  (months). 
This  said,  in  our  experimental  work,  the  purely  mathematical  use  of 
BCOT  did  remarkably  well. 

Figure  5.8  illustrates  the  sensitivities  mentioned  above.  Each 
group  of  bars  corresponds  to  one  of  the  dominant  points  in  Figure 
5.7*  However,  Figure  5.8  shows  the  effectiveness  of  each  such  point 
(option)  for  the  baseline  versions  of  the  three  representative  scenarios. 
In  this  case  (i.e.,  with  the  particular  assumptions  used  to  generate  the 

Figure  5.8 

Sensitivity  of  Dominant-Point  Effectiveness  to  Scenario 


RAND  MG662-5.8 


Apportioned  costs  (Smillions) 


This  is  not  immediately  obvious  because  of  differences  in  scale,  but  the  reader  can  at  least 
verify  that  there  are  13  dominant  points  in  Figure  5.7  and  13  groups  of  bars  in  Figure  5.8. 
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chart),  most  of  the  dominant-point  options  are  in  fact  quite  ineffective 
for  attacking  mobile-missile  targets  (scenario  class  1).  With  a  different 
set  of  equally  reasonable  assumptions,  the  dominant  points  themselves 
would  be  different,  and  it  might  be  that  another  scenario  class  would 
not  be  dealt  with  effectively. 

Dealing  properly  with  such  large  and  complicated  effects  of 
uncertainty  requires  a  new  version  of  exploratory  analysis  that  we  are 
in  the  process  of  developing.  In  the  meantime,  we  used  more-classical 
methods  of  sensitivity  analysis  that  provided  insights,  although  they 
were  not  entirely  satisfying.  Table  5.7  lists  the  parameter  variations  that 
we  used  in  our  analysis. 

A  fuller  exploratory  analysis  would  examine  results  throughout 
the  parameterized  space.  Among  other  things,  it  would  include  finding 
results  for  all  combinations  of  the  various  excursions  in  Table  5.7,  as 
well  as  varied  assumptions  about  all  the  other  factors  mentioned  earlier 
(focus,  building-block  costs,  and  so  on).  In  some  problems,  that  kind  of 
extended  exploratory  analysis  would  reveal  issues  not  previously  appre¬ 
ciated;  in  others,  it  would  not.  A  major  characteristic  of  good  analysts, 
of  course,  is  that  they  are  able  to  identify  the  key  factors  intuitively, 


Table  5.7 

Illustrative  Broad-Ranging  Sensitivity  Analysis  During  Screening 


Sensitivity 

Values  Used  in  Excursions 

Performance  of  automated-target- 
recognition  (ATR)  systems 

False-alarm  rates  higher  and  lower  than 
baseline  value  by  a  factor  of  10 

Level  of  air  defense 

Penetration  probabilities  for  advanced, 
double-digit  SAMs,  with  and  without 
networking 

Vulnerability  of  buried  targets 

Effects  of  uncertainty  in  underground 
locations,  hardness,  and  potential 
effectiveness  of  joint-attack  mechanisms 

Performance  of  advanced  bomber 

Degrees  of  stealth  (as  proxied  by  ability  to 
defeat  advanced  air  defenses) 

Sensitivity  to  assumed  suppression 
attacks 

Consequences  of  not  having  initial 
defense-suppression  attacks  in  mobile- 
missile  and  WMD-facilities  scenario 
classes 

Cost-sharing  for  advanced  bomber 

Substantial  increase  in  Global  Strike's 
share  of  cost  (25%  or  50%,  rather  than 
the  baseline  value  of  10%) 
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without  the  benefit  of  the  more-systematic  exploration  that  is  only  now 
becoming  possible  through  the  use  of  advanced  computers  and  com¬ 
putational  methods. 

Without  further  elaboration,  the  set  of  options  that  passed  screen¬ 
ing  and  seemed  to  us  appropriate  for  a  next  phase  of  analysis  in  a  fuller 
portfolio  framework  is  shown  in  Table  5.8. 

As  suggested  by  our  discussion  above,  some  of  the  options  were 
expected  to  look  good  only  in  some  scenario  classes,  or  even  in  par¬ 
ticular  cases  within  those  scenario  classes.  Others  were  seen  as  more 
generally  useful. 

A  basic  issue  in  evaluating  an  option-generation-and-screening 
tool  such  as  BOOT  is  whether  the  results  highlight  options  that  would 
have  been  missed  otherwise.  Our  experience  is  that  additional  options 
do  turn  up  as  a  consequence  of  using  such  a  tool.  Afterward,  it  may  be 
“obvious”  that  those  options  should  have  been  considered,  but  intu¬ 
ition  is  notoriously  discontinuous  (what  is  obvious  afterward  was  not 
obvious  before).  The  value  of  the  approach  is  especially  high  when  there 
are  multiple  objectives  or,  as  in  the  Global  Strike  example,  when  there 
are  multiple  scenarios  with  which  to  deal. 

Table  5.8 

Screened  Composite  Options  for  Portfolio  Analysis 

Forw.  basing 
Penaids 

SLBM  +  penaids 
Sensors 

SLBM  +  penaids  +  SOF  vehicle 

SLBM  +  penaids  +  sensors 

SLBM  +  penaids  +  sensors  +  SOF  vehicle 

Forw.  basing  +  sensors  +  SOF  vehicle 

Forw.  basing  +  penaids  +  sensors  +  SOF  vehicle 

Adv.  bomber  +  sensors 

Adv.  bomber  +  sensors  +  SOF  vehicle 

Forw.  basing  +  adv.  bomber  +  sensors  +  SOF  vehicle 

ICBM  +  SLBM  +  adv.  bomber  +  penaids  +  SOF  vehicle 
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Framework  for  Portfolio  Analysis 

As  mentioned  earlier,  we  have  used  a  portfolio-analysis  tool  to  orga¬ 
nize  information  and  generate  displays  appropriate  for  review  meet¬ 
ings.  Table  5.9  shows  a  schematic  of  the  summary  display  at  the 
beginning  of  our  work.  Options  are  in  rows  (Base,  1,  2,  3),  with  the 
baseline  option  (no  changes  in  programs)  as  the  first  option  to  be  eval¬ 
uated.  Columns  provide  different  measures  of  the  options’  goodness. 
The  scorecard  region  provides  a  quick  overview.  The  area  to  the  right 
includes  columns  with  numbers  rather  than  colors — e.g.,  the  options’ 
costs,  a  net  effectiveness  computed  over  the  various  evaluation  cases  in 
the  scorecard,  and  a  relative  measure  of  benefit  divided  by  cost.  Often, 
we  don’t  show  these  columns  at  the  same  time  we  show  those  of  the 
effectiveness  scorecard. 

As  discussed  earlier,  we  concluded  that  for  summary  purposes, 
it  was  sufficient  to  show  effectiveness  in  each  of  two  cases  for  each  of 
three  scenario  classes. 

Summary  (Notional)  Results  for  Effectiveness 

Figure  5.9  is  a  screenshot  of  the  scorecard  portion  of  the  summary  sheet 
for  our  illustrative  analysis.  The  results  shown  here  are  notional  and 

Table  5.9 

Schematic  Representation  of  Displays  to  Follow 


Option 
Base 
1 
2 
3 

NOTE:  Risk  =  a  measure  of  net  risk  across  the  scenario  classes;  cost  =  dollar  cost,  as 
in  present  value  of  life-cycle  cost;  net  effectiveness  =  net  effectiveness  across  the 
measures  of  option  goodness;  RCE  =  relative  cost-effectiveness. 


Measures  of  Option  Goodness 

Cost 

Net 

Effectiveness 

RCE 

Effectiveness  by  Scenario  Class 

Mobile  Missiles  Terrorists  WMD  facilities 

A 

B 

A 

B 

A 

B 

Risk 
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Figure  5.9 

Summary  Scorecard 


Options 


Measures  of  Option  Goodness:  Effectiveness  (Color) 
by  Scenario  Class  and  Overall  Risk 


Measures 


Mobile 

Missiles 

(2020) 


Mobile 

Missiles 

(2020) 

Reactive 

Threat 


Terronst 

Site 

(2020) 


Terrorist 

Site 

(2020) 

Fleeting 

Target 


WMD 

Facilities 

(2020) 


WMD 

Facilities 

(2020) 

Hard 

Case 


Overall 

Risk 

(2020) 


Investment  Options 


Base  Case 


Sensors 


Forw.  Basing 


Penaids 


Forw.  Basing  + 
Penaids  +  Sensors 


Penaids  +  Sensors  +| 
SOF  Vehicle 


SLBM  +  Penaids 


Adv.  Bomber  + 
Sensors 


SLBM  +  Penaids  + 
Sensors 


SLBM  +  Penaids  + 
SOF  Vehicle 


Sensors  +  SOF 
Vehicle 


Sensors  +  SOF 
Vehicle 


Bomber  +  Sensors  + 
SOF  Vehicle 


NOTE:  Effectiveness  colors:  red,  orange,  yellow,  light  green,  and  green  indicate  very 
poor,  poor,  marginal,  good,  and  very  good.  Options  are  listed  in  increasing  order  of 
cost.  An  (F)  indicates  that  the  option  failed  a  performance  threshold. 
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somewhat  contrived — they  do  not  reflect  accurate  data  on  the  options. 
The  options  appearing  in  the  rows  are  those  identified  by  the  screening 
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as  worthy  of  detailed  analysis.  PAT  can  order  the  options  in  a  variety 
of  ways,  but  in  Figure  5.10,  they  are  ordered  by  effectiveness,  with  the 
least- effective  option  (the  baseline)  on  top. 

Although  it  is  not  evident  from  Figure  5.9,  the  results  shown  for 
some  of  the  options  assume  force  employment  based  on  naval  aviation 
from  carrier  battle  groups;  other  options  employ  the  advanced  bomber; 

Figure  5.10 

Zooming  to  Understand  the  Basis  of  Top-Level  Assessments 


Forw.  Basing  +  Adv.  Bomber 
Sensors  +  SOF  Vehicle 


NOTE:  Effectiveness  colors:  red,  orange,  yellow,  light  green,  and  green  indicate  very 
poor,  poor,  marginal,  good,  and  very  good.  Timeliness  is  indicated  in  hours;  execution 
and  operational  effects  are  measured  in  probabilities  of  success. 
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still  others  employ  ballistic  missiles;  and  so  on.  For  each  row,  the  results 
shown  are  for  the  most  effective  force-employment  mechanism  avail¬ 
able  within  the  baseline  program,  supplemented  by  what  the  particu¬ 
lar  option  procures.  That  is,  the  analysis  was  based  on  optimized  force 
employment  (within  the  possibilities  specified  as  CONOPS).  In  some 
cases,  the  most  effective  force  employment  in  an  option  is  to  use  only 
baseline  capabilities. 

Zooming  (Drilling  Down)  for  Explanation 

According  to  the  notional  analysis,  projected  baseline  capability  is 
rather  poor,  although  the  United  States  retains  some  capability  against 
the  terrorist-like  class  of  targets.  Successive  rows  in  Figure  5.9  even¬ 
tually  show  relatively  good  results  for  attacking  terrorists  and  WMD 
facilities,  but  mediocre  results  for  attacking  mobile  missiles,  if  the 
SLBM  option,  penaids  for  tactical  aircraft,  sensors,  and  a  SOF  vehicle 
are  added.  One  noteworthy  point  is  that  some  options — as  expected — 
improve  results  for  one  class  of  scenarios,  but  not  for  another  (as  in 
Figure  5.9). 

Suppose,  now,  that  we  ask  for  an  explanation  of  results  for  the 
first  scenario  class,  the  class  involving  mobile  missiles.  Figure  5.10 
shows  the  result  of  zooming  on  the  second  column  of  the  scorecard 
in  Figure  5.9  (the  column  for  mobile  missiles  (2020)).  That  column  is 
reproduced  as  the  rightmost  column  in  Figure  5.10,  i.e.,  it  is  the  result 
of  combining  the  results  on  the  left  side  of  the  table.  We  see  that  the 
evaluation  in  Figure  5.10  is  an  aggregation  based  on  considering  the 
timeliness  of  attack,  the  effectiveness  of  the  attack’s  execution  itself, 
and  an  assessment  of  likely  operational  effects  (i.e.,  whether  the  attack 
would  accomplish  the  larger  military  purpose).  According  to  Figure 
5.10,  results  for  the  first  and  third  of  these  are  very  good  (green),  but 
the  ability  to  execute  the  attack  is  very  bad  (red)  in  many  cases — except 
in  those  that  include  procurement  of  the  advanced  sensors,  as  well  as 
either  an  advanced  bomber  or  penetration  aids.  That  is,  in  this  sce¬ 
nario,  the  United  States  has  difficulty  penetrating  to  the  target  area 
and  loitering  long  enough  to  search  for  targets,  and  it  lacks  the  sensor 
capabilities  permitting  that  search. 
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In  this  and  other  zooms,  the  issue  of  how  we  aggregate  upward 
arises.  That  is,  given  the  component  values  in  Figure  5.10,  how  do  we 
calculate  the  net  result?  The  question  is  not  trivial,  and  a  significant 
part  of  our  effort  went  into  thinking  through  the  related  theory. 

The  answer  is  that  the  aggregation  rule  depends  on  the  particular 
problem  addressed.  It  can  be  as  simple  as  an  average  or  a  linear-weighted 
sum.  Or  it  may  be  more  complicated,  as  in  evaluating  the  whole  as  the 
worst  value  of  its  components.  In  still  other  cases,  as  shown  in  what 
follows,  we  specified  a  combining  rule  corresponding  to  the  mathemat¬ 
ics  of  combining  probabilities,  e.g.,  the  probability  of  A  and  B  is  the 
product  of  the  probabilities  for  A  and  B  separately.  Specifying  the  com¬ 
bining  rules  is  quite  important  and  therefore  not  something  to  be  hard¬ 
wired  into  a  tool  (as  is  common  in  commercial  decision-support  tools, 
which  assume  aggregation  by  linear  weighted  sums,  ignoring  impor¬ 
tant  effects  of  nonlinearity). 

If  we  wish  to  find  out  why  execution  failed,  we  can  zoom  again, 
this  time  to  the  display  shown  in  Figure  5.11.  Because  we  assumed 
that  the  problem  of  killing  the  target  was  trivial  if  the  target  could 
be  found,  the  calculation  depends  only  on  penetration  and  fmd-and- 
kill  probabilities.  The  primary  problem  is  in  the  find-and-kill  column. 
The  figure  indicates  that  many  of  the  options  have  extreme  problems 
finding  and  killing  the  targets — even  if  they  are  able  to  penetrate  air 
defenses  successfully,  as  options  with  SLBMs  or  ICBMs  could.  This 
merely  reflects  the  absence — in  currently  projected  forces — of  good 
capabilities  for  searching  areas  to  find  mobile  targets.  The  combined 
effectiveness  is  then  calculated  as  the  product  of  two  probabilities,  and 
it  is  shown  to  be  bad.  Visually,  this  means  that  the  result  of  a  red  and 
a  green  is  not  yellow,  but  red.  The  figure  indicates  that  with  a  combi¬ 
nation  of  advanced  sensors,  the  finding-and-killing  problem  can  be 
resolved,  and  with  the  addition  of  an  advanced  bomber  or  penaids,  the 
mission  can  be  accomplished  rather  well  (subject  to  assumptions  in  the 
default  set  of  parameter  values  for,  e.g.,  the  capabilities  of  advanced  air 
defense,  search  capability  with  a  combination  of  advanced  sensors,  and 
so  on).  The  last  column  is  necessary  for  explanation  if  the  user  has  iden¬ 
tified  minimum  levels  of  effectiveness  that  are  to  be  given  any  credit — 
i.e.,  if  the  user  has  identified  thresholds.  In  that  case,  the  results  in  the 
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Figure  5.11 

Zooming  Further  to  Understand  Assessment  of  Execution 
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next-to-last  column  are  rescaled  before  being  shown  at  the  higher  level 
(in  this  discussion,  Figure  5.10).  For  this  example,  we  set  a  threshold 
of  0.5. 
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Zooming  to  a  Level  of  Systems  Analysis 

Occasionally,  it  will  be  necessary  to  go  even  deeper  to  understand  a 
high-level  result.  The  next  zoom  in  our  methodology  is  different  in 
kind  from  the  previous  zoom.  Its  intent  is  to  summarize  systems  analy¬ 
sis.  The  appropriate  zoom  depends  on  the  options  we  are  looking  at. 
Let  us  ask  why  locating  and  killing  targets  might  be  enhanced  by 
automatic-target  recognition,  a  satellite  constellation  for  persistent  sur¬ 
veillance,  and  defense  suppression. 

The  displays  in  Figure  5.12  are  in  the  natural  language  of  systems 
analysis — charts,  rather  than  top-level  management  spreadsheets.  The 
three  panels  show  what  we  might  see  at  that  level,  representative  para¬ 
metric  plots.  The  top  left  panel  explains  that  search  rate  drops  rapidly  as 
the  resolution  of  a  search  radar  increases.  The  top  right  panel  explains 
that  the  false-alarm  rate  is  quite  high  for  low-resolution  radars  but  falls 
by  orders  of  magnitude  with  increased  radar  resolution.  Putting  these 
two  conflicting  considerations  together  with  simple  mathematics,  one 
can  estimate  the  probability  of  finding  (and  killing)  a  mobile  target 
such  as  a  mobile  ICBM.  The  calculation  depends  on  the  search  area 
(which  might  be  reduced  with  the  existence  of  space-based  synthetic- 
aperture  radars),  the  search  radar’s  resolution,  and  the  presence  on  attack 
aircraft  of  advanced  signal-processing  technology  such  as  automated 
target  recognition  for  reducing  the  number  of  false  targets.  The  bottom 
panel  summarizes  this  calculation,  indicating  that  whereas  currently 
programmed  capabilities  would  be  poor,  even  if  the  search  area  were 
low,  it  is  possible  with  advanced  capabilities  represented  in  some  of  the 
options  to  achieve  better  results  if  the  search  area  is  small  enough — as 
it  might  be  as  the  result  of  pre-attack  intelligence  and  advanced  sur¬ 
veillance.  This  assessment  is  sensitive  to  assumptions  and  requires 
engineering-level  analysis.57  As  a  sobering  calibration  point,  in  Opera¬ 
tion  Iraqi  Freedom  the  search  area  was  in  the  high  range  of  Figure  5.12. 
It  should  not  have  been  surprising  that  the  famous  “Scud  ITunt”  was  a 
failure. 
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Figure  5.12 

Zooming  to  the  Level  of  Systems  Analysis 


Area  to  be  searched  (square  km) 


Another  sobering  observation  is  that  the  competition  of  measures 
and  countermeasures  is  always  an  issue.  If  the  United  States  deployed 
the  advanced  sensors  and  ATR  capabilities  allowing  it  to  find  mobile 
missiles,  the  adversary  would  employ  countermeasures,  and  the  United 
States  would  then  have  to  improve  its  capabilities  further  to  counter 
those  countermeasures. 
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How,  then,  do  we  assess  the  capability  of  such  advanced  sys¬ 
tems  amid  uncertainty?  Assuming  success  of  technology,  effectiveness 
would  be  high  in  the  absence  of  countermeasures  and  low  or  mar¬ 
ginal  if  countermeasures  were  present.  We  don’t  want  to  “average” 
those  cases,  so  instead  we  show  results  with  and  without  the  successful 
countermeasures. 

A  Final  Example  of  Zooming 

We  present  one  more  example  to  show  how  zooming  can  be  impor¬ 
tant  to  understanding  and  using  the  portfolio-analysis  tool.  Looking 
back  at  the  last  column  of  Figure  5.10,  we  see  that  results  for  attacking 
WMD  facilities  are  generally  mediocre  to  poor.  That  may  seem  coun¬ 
terintuitive,  because  “surely”  the  United  States  will  have  the  ability 
to  penetrate  air  defenses  and  deliver  precision  munitions  against  such 
facilities.  Figure  5.13,  however,  shows  that  at  the  next  level  of  detail 
(not  actually  very  much  detail  in  this  case),  the  collateral  damage  factor 
is  poor  (orange).  The  reason  is  that  if  intelligence  were  fairly  poor  and 
the  facilities  had  to  be  attacked  by  massive  bombing  alone,  even  with 
precision  weapons  there  would  be  (according  to  this  notional  analysis) 
a  substantial  chance  of  releasing  toxic  chemicals,  biological  materials, 
or  even  radioactive  material.  This  problem  did  not  arise  in  the  first  of 
the  two  WMD-facilities  cases  because  it  was  assumed  that  the  facilities 
could  be  appropriately  destroyed,  damaged,  or  put  out  of  commission 
without  such  indiscriminate  bombing — perhaps  through  joint  opera¬ 
tions  involving  bombers,  missiles,  intelligence,  and  SOF  units.  Had 
we  considered  cyberspace  options,  those  might  also  have  played  a  role 
here. 

Figure  5.13  also  illustrates  another  mechanism  that  we  use  to 
highlight  fragile  assumptions  that  might  otherwise  be  hidden.  Figure 
5.9  contained  small  red  triangles  (warning  indicators)  in  many  of  the 
cells.  By  mousing  over  these  triangles,  the  PAT  user  will  see  a  brief  tex¬ 
tual  warning  message.  The  warning  column  in  Figure  5.13  shows  the 
text  itself. 
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Figure  5.13 

Explanation  of  Poor  Results  Against  WMD  Facilities 
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corresponding  to  highly  notional  probabilities.  Timeliness  is  measured  in  hours. 
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Summary  Results  for  Cost-Effectiveness  Comparisons 

Traditionally,  a  key  aspect  of  defense-planning  analysis  has  been  the 
economic  analysis:  How  much  payoff  can  be  obtained  as  a  function  of 
expenditure? 

Figure  5.14  shows  an  illustrative  bottom-line  display  of  effective¬ 
ness  versus  costs  for  the  various  options.*  Such  displays  can  be  readily 
generated  with  PAT.  Each  dot  on  the  chart  represents  a  different  invest¬ 
ment  option.  The  dots’  colors  have  no  meaning  other  than  to  distin¬ 
guish  among  options.  For  the  particular  set  of  assumptions  underlying 
Figure  5.14,  an  attractive  point  might  be  procuring  the  SLBM,  pene¬ 
tration  aids,  sensors,  and  a  SOF  vehicle.  Even  more  effectiveness  can  be 

Figure  5.14 

Cost-Effectiveness  for  a  Baseline  Analysis 
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The  astute  reader  may  be  asking  why  this  chart  does  not  look  more  regular,  with  more- 
expensive  options  always  being  better.  The  reason  is  that  the  options  plotted  showed  up  as 
“good”  (i.e.,  at  or  near  an  efficient  frontier)  in  some  of  the  cases  used  for  screening,  but  not 
necessarily  on  average. 
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achieved  with  the  buy-everything  option,  which  includes  an  advanced 
bomber  and  a  ground-based  ICBM.*  However,  those  additions  cost  a 
good  deal  more  money  for  little  additional  benefit. 

Unfortunately,  but  consistent  with  a  continuing  theme  of  this 
monograph,  it  is  impossible  to  draw  valid  conclusions  from  any  one 
such  chart;  the  results  depend  far  too  sensitively  on  assumptions  for 
such  “mechanical”  analysis  to  be  useful.  Notions  of  optimization  are 
inappropriate. 

Figure  5.15  compares  similar  plots  but  with  different  relative 
emphases  on  the  six  scenario  cases  that  we  have  used.  The  first  plot 
(top  left)  is  the  same  plot  shown  in  Figure  5.14  and  corresponds  to 
equal  weighting  of  the  six  scenarios.  The  next  plot  (top  right)  corre¬ 
sponds  to  giving  equal  weight  to  the  three  less-stressful  versions  of  the 
scenarios.  It  is,  more  or  less,  a  best  case  for  the  options,  because  the 
various  improvement  options  all  have  substantial  effects,  and  the  more 
one  spends,  the  better  the  overall  results  turn  out  to  be.  In  contrast, 
the  bottom  left  figure  corresponds  to  giving  equal  weight  to  the  three 
most-stressful  versions  of  the  scenarios  (those  that  include  countermea¬ 
sures — the  second,  fourth,  and  sixth  cases).  Here,  the  plot  of  effective¬ 
ness  versus  cost  is  more  nearly  flat — except  for  a  significant  payoff  at 
very  low  cost.  The  “smart”  option  would  be  to  buy  just  the  Trident 
Conventional  Missile  and  penetration  aids. 

Finally,  the  bottom  right  plot  shows  results  if  only  the  terrorist 
and  WMD-facility  scenarios  are  considered  and  the  four  correspond¬ 
ing  cases  are  equally  weighted.  That  is,  we  discount  the  mobile-missile 
scenario  (perhaps  because  of  a  judgment  that  its  success  is  too  dubious 
and  that  type  of  global  strike  would  be  very  risky).  In  this  case,  we  see 
that  a  “smart”  option  might  be  to  proceed  with  the  SOF  insertion  vehi¬ 
cle,  as  well  as  the  Trident  Conventional  Missile  and  penetration  aids. 

With  our  apologies  for  redundancy,  we  emphasize  yet  again  that 
these  results  are  all  notional  and  would  change  in  a  more-complete  analy- 


In  our  illustrative  analysis,  the  ICBM  provided  no  more  than  the  Trident  Conventional 
Missile  (TCM),  but  at  a  much  higher  cost.  A  more  complete  analysis  would  have  included 
measures  giving  more  value  to  the  ICBM  for  its  payload  and  growth  potential  and  its  ability 
to  avoid  overflying  countries  of  concern.  Such  a  boost-glide  capability,  however,  would  entail 
considerable  development  risks. 
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Figure  5.15 

Cost-Effectiveness  Plots  for  Different  Assumptions 
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sis  or  with  more-accurate  data.  In  particular,  results  would  change  if, 
for  example,  we  (1)  discounted  the  need  for  one-hour  execution  times 
because  other  factors  were  dominating,  (2)  discounted  the  feasibility  of 
achieving  adequate  penetration  probabilities  by  merely  adding  penetra- 
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tion  aids  to  current  aircraft,  (3)  assumed  more-  or  less- effective  intel¬ 
ligence,  surveillance,  and  reconnaissance  (ISR),  and  so  on. 

Despite  our  trepidation  at  doing  so  because  of  the  notional  char¬ 
acter  of  our  data,  we  will  draw  some  tentative  conclusions  later  in  this 
chapter,  but  they  will  be  conclusions  that  appear — without  the  benefit 
of  more-detailed  analysis — to  be  relatively  robust.  Before  that,  how¬ 
ever,  let  us  touch  upon  the  problem  of  interactions  across  capability 
areas  and  assessments  of  risk. 

Finding  Tradeoffs  and  Bill-Payers  Within  a  Capability  Area 

DoD  policymakers  routinely  lament  the  difficulty  they  have  in  finding 
recommendations  for  cutting  back  on  one  investment  to  help  pay  for 
another.  Such  cutbacks  may  be  referred  to  as  “taking  more  risks  in  one 
area  in  order  to  reduce  risks  in  another,”  or  as  “recognizing  where  we 
have  an  excess  of  capability.”  It  is  unnatural  for  organizations  to  vol¬ 
unteer  recommendations  about  where  cuts  can  be  made  in  their  own 
domains.  It  usually  proves  more  productive  to  give  them  a  budget  and 
force  them  to  work  within  it.  They  will  then  make  their  own  tradeoffs, 
drawing  on  their  in-depth  knowledge. 

This  situation  has  led  to  the  frequent  suggestion  that  those  respon¬ 
sible  for  capability  areas  should  seek  to  balance  their  portfolios  sepa¬ 
rately  for  those  areas.  That  can  sometimes  be  feasible  and  useful  (put¬ 
ting  aside  issues  of  organizational  structure  and  power).  However,  our 
work  on  Global  Strike  has  demonstrated  clearly  why  such  a  strategy  is 
unworkable  in  many  cases  of  interest. 

The  core  problem  is  that  capability  areas  interact:  A  given  system, 
such  as  a  potential  constellation  of  space-based  synthetic-aperture 
radars  (SARs)  or  a  new  long-range  bomber,  would  contribute  substan¬ 
tially  to  multiple  capability  areas.  This  implies  that  to  do  cost-benefit 
analysis  within  a  given  area,  one  must  use  the  fractional  costs  allocated 
to  that  area,  as  in  assuming  that  Global  Strike  will  pay  for,  say,  10 
percent  of  a  SAR  constellation.  But  who  does  the  allocation,  and  what 
if  the  number  later  changes?  It  is  one  thing  to  have  uncertainty  about 
a  system  largely  associated  with  the  area  in  question  (e.g.,  a  20  per¬ 
cent  uncertainty  in  the  cost  of  a  new  conventional  ICBM,  all  of  which 
should  be  borne  by  the  Global  Strike  capability  area),  but  the  uncer- 
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tainty  in  the  cost  to  Global  Strike  might  be  a  factor  of  two  or  more  for 
something  like  the  space-based  SARs.  Why?  Because  the  10  percent 
figure  would  be  substantially  arbitrary,  the  result  of  negotiations  and 
gamesmanship.  Consequently,  it  might  be  doubled  at  some  later  date. 
Another  aspect  of  this  problem  is  that  the  real  issue  is  cross-cutting: 
Is  there  enough  cumulative  demand  for  the  system  at  issue,  demand 
across  capability  areas,  to  justify  its  procurement?  A  given  capability 
area’s  “manager”  cannot  by  himself  decide  to  buy  or  forgo  the  system 
unless  his  capability  area  will  carry  the  burden  of  the  expense. 

Another  economic  problem  is  that  two  capabilities  may  have  com¬ 
parable  costs  but  may  arrive  years  apart,  even  a  decade  or  more  apart. 
Comparisons  are  then  difficult,  even  though  using  net  present  values 
is  a  well-established  method  in  economics.  Unlike  in  the  business 
world,  there  is  no  clear-cut  discount  rate  to  be  used  for  defense-system 
procurements.  Further,  it  may  be  more  important  that  one  capability 
arrives  later  than  the  other  than  that  it  costs  more  or  less. 

These  problems  are  by  no  means  insurmountable.  In  a  particular 
case,  it  will  be  necessary  to  step  outside  the  individual  capability  area 
and  have  a  cross-area  decision  process  focused  on  the  particular  can¬ 
didates  of  interest.  Figure  5.16  illustrates  the  point  schematically.  Sup¬ 
pose,  for  example,  the  Global  Strike  analysis  is  uncertain  because  it  is 
not  known  how  much  of  the  cost  of  some  systems  (e.g.,  a  constellation 
of  SARs)  should  be  allocated  to  the  Global  Strike  capability  area.  The 
issue  cannot  be  resolved  “inside”  the  Global  Strike  domain.  Instead,  a 
side  assessment  would  be  needed  that  would  consider  the  other  capa¬ 
bility  areas  to  which  the  system  in  question  would  contribute.  Judg¬ 
ments  would  be  needed  as  to  how  overall  costs  would  be  allocated 
across  areas.  Some  pointed  research  is  needed  to  clarify  precisely  what 
the  process  shown  would  require,  analytically  and  otherwise. 

Treatment  of  Risk 

Although  we  did  not  discuss  the  risk  issue  earlier,  it  was  folded  into  the 
portfolio  analysis  shown  in  Figures  5.10  onward.  Here  we  discuss  in 
more  detail  what  we  did  to  address  risk,  making  points  that  are  rela¬ 
tively  generic. 
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Figure  5.16 
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There  are  many  different  kinds  of  risk,  some  of  which  cannot  be 
measured  with  accuracy  and  precision  or  even  quantified  in  the  usual 
sense.  The  most  important  of  these  should  be  represented  in  the  analy¬ 
sis,  even  if  their  basis  is  subjective  and  qualitative.  It  is  important,  how¬ 
ever,  to  define  clearly  what  is  being  assessed  and  why  the  judgments 
expressed  were  used.  This  constitutes  a  form  of  rigor  that  should  not  be 
underestimated.  No  policymaker  likes  to  be  shown  colored  scorecards 
dominated  by  mysterious  subjective  judgments  arrived  at  by  voting 
schemes,  especially  when  the  voters  are  a  miscellany  of  people  who 
typically  do  not  see  the  world  the  way  the  policymakers,  whether  gen¬ 
eral  officers  or  civilians,  do.  The  policymakers  want  to  know  who  made 
the  judgments  (by  name)  and  why.  Thus,  explanation  is  important. 

Types  of  Risk 

In  our  application  to  Global  Strike,  we  created  placeholders  for  the  fol¬ 
lowing  types  of  risk,  as  indicated  in  Figure  5.17. 
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Figure  5.17 

Risk  Hierarchy  Used  in  the  Global  Strike  Application 
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Political  and  Strategic  Risk  During  Development.  How  likely  is  it 
that  the  program  will  be  rejected  by  Congress  or,  if  it  depends  on  bases 
or  other  allied  cooperation,  by  the  relevant  friendly  governments?  Even 
if  there  is  initial  assent,  will  it  persist  over  the  longer  term?  Such  risks 
for  Global  Strike  arise  with  respect  to  forward  basing  of  aircraft  and 
the  staging  of  SOF  operations.  It  might  be  argued  that  such  “political” 
issues  have  no  role  in  analysis,  but  we  disagree  to  the  extent  that  there  is 
a  sound  basis  for  distinguishing  among  otherwise  similar  options  with 
low,  moderate,  or  high  risks  of  this  type.  Within  the  realm  of  Global 
Strike,  an  example  of  political  risk  is  that  Congress  will  refuse  to  fund 
certain  prompt  Global  Strike  options  (such  as  the  Conventional  Tri¬ 
dent  Modification)  because  of  concerns  about  a  launch  being  mistaken 
for  a  nuclear  attack,  resulting  in  a  nuclear  response  being  made  against 
the  United  States.* 


The  Senate  Appropriations  Committee  has  expressed  concerns  about  this  and  other  mat¬ 
ters.  A  recent  National  Academy  study  describes  the  concerns  and  its  own  assessment.  It 
recommends  doing  full  research,  development,  and  testing  of  the  CTM  but  deferring  deci- 
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Technical  and  Programmatic  Risks.  How  likely  is  technical  failure 
to  achieve  the  posited  capability,  and  how  does  that  likelihood  vary 
with  time  and  expense?  Some  options  are  uncertain  only  in  that  times 
may  slip  and  costs  may  go  up;  in  other  cases,  it  is  not  clear  that  the 
problems  of  physics  and  engineering  can  be  solved;  in  still  other  cases 
(not  so  much  with  Global  Strike  options),  it  may  be  unclear  whether 
the  industrial  base  is  able  to  accomplish  what  is  needed.58  For  Global 
Strike,  achieving  adequately  good  automated  target  recognition  involves 
considerable  technical  and  programmatic  risk,  especially  in  areas  with 
complex  terrain  or  considerable  clutter.59 

Strategic  Risks  at  the  Time  of  Execution.  How  likely  is  it  that 
the  posited  capability  would  be  usable  when  needed?  Again,  there  are 
issues  about  both  internal  political  and  strategic  judgments  and  what 
allies  would  permit  or  tolerate.  The  flexible-response  options  of  the 
1970s  and  1980s  were  highly  controversial  but  were  deemed  relatively 
credible  because  NATO  took  extraordinary  efforts  to  implement  rel¬ 
evant  strategic,  operational,  and  tactical  doctrines. 

Another  source  of  strategic  risks  at  the  time  of  execution  could 
be  a  situation  in  which  other  military  actions  are  tying  up  U.S.  forces 
associated  with  the  option  in  question.  For  example,  many  support¬ 
ing  resources  would  probably  be  deemed  necessary  to  achieve  penetra¬ 
tion  of  air  defenses,  even  if  those  resources — e.g.,  tactical  aircraft  of 
various  types,  tankers,  aircraft  carriers,  national  C4ISR  assets — were 
fitted  with  penetration  aids.  “Maldeployment”  and  simply  being  oth¬ 
erwise  occupied,  then,  are  strategic  risks  affecting  the  various  options 
differently. 

Another  class  of  strategic  risks  could  arise  if  a  strategic  strike  with 
conventional  weapons  were  misperceived  as  involving  nuclear  weap¬ 
ons.  This  concern  has  been  raised  with  respect  to  using  Trident  or  Min- 
uteman  missiles  for  conventional  strikes,  although  the  issue  is  more 
generic. 


sions  about  deployment  until  test  results  are  in  and  the  issues  are  more  fully  analyzed.  (See 
National  Research  Council,  2007.)  The  House  Armed  Services  Committee  supported  the 
CTM  option  as  of  summer  2007. 


A  Notional  Example:  Global  Strike  95 


A  final  example  of  this  class  of  risks  involves  the  Bush  administra¬ 
tion’s  intention  to  deploy  ballistic-missile  defenses  in  Eastern  Europe. 
This  issue  has  become  highly  controversial,  antagonizing  to  the  Rus¬ 
sians  and  in  conflict  with  current  public  opinion  in  most  of  Europe. 
Attitudes  on  the  matter  could  shift  and  shift  again  in  the  years  ahead, 
depending  on  political  developments  in  Iran  and  elsewhere. 

Availability.  A  risk  common  to  all  of  the  options  but  of  more 
importance  in  some  of  the  test  cases  than  others,  is  the  risk  that  the 
option’s  preferred  instrument  for  a  given  mission  (the  mission  of  one  of 
the  test  scenarios)  will  not  be  available  when  it  is  needed.  Possible  rea¬ 
sons  for  such  unavailability  include  (1)  not  enough  warning  time,  (2) 
maldeployment  (e.g.,  a  carrier  battle  group  may  be  hundreds  of  miles 
from  where  it  needs  to  be),  and  (3)  an  ongoing  operation  (e.g.,  a  sepa¬ 
rate  crisis  or  even  a  major  exercise).  Many  other  possibilities  also  exist. 

Analyses  of  such  issues  are  interesting  and  in  some  respects 
counterintuitive.  Skeptics  of  “prompt  global-strike  options”  such  as 
the  Conventional  Trident  Modification  (CTM)  or  other  ballistic- 
missile  approaches  tend  to  note  that  substantial  times  would  be  needed 
for  alerting  and  maneuvering  forces,  conducting  surveillance,  target¬ 
ing,  mission  planning,  and  especially  decisionmaking.  Therefore,  they 
argue,  there  would  be  little  value  in  an  option  that  improved  execution 
time  a  bit  (e.g.,  reducing  it  from  several  hours  to  one  hour).  Execution 
time  would  be  a  small  fraction  of  the  total  time.  The  argument,  how¬ 
ever,  is  flawed.  First,  as  discussed  earlier  in  this  chapter,  historical  and 
prospective  cases  exist  in  which  execution  time  is  indeed  the  limiting 
factor  because  of  prior  intelligence  and  decisionmaking.  Second,  even 
if  some  hours  are  available  and  needed  for  decisionmaking  and  other 
preparations,  it  may  very  well  be  that  platforms  such  as  an  aircraft  car¬ 
rier  are  in  the  midst  of  operations  hours  away  from  where  they  need 
to  be.  The  operation  may  also  depend  on  tanker  aircraft  positioning 
themselves  appropriately.  Thus,  the  timelines  shown  early  in  this  chap¬ 
ter  for  tactical  aircraft  are  optimistic.  Considering  such  risks  increases 
the  relative  attractiveness  of  ballistic-missile  options  (because  of  their 
speed  and  readiness)  and  perhaps  cruise-missile  options  (because  the 
launch  platforms  may  be  less  likely  to  be  maldeployed).  It  also  makes 
the  long-range-bomber  option  less  unattractive — that  is,  it  competes 
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well  if  something  on  the  order  of  10  or  12  hours  is  needed  for  all  the 
preparations. 

We  did  nor  analyze  availability  issues  in  any  depth.  However,  in 
a  real-world  study,  we  would  have  done  so.  We  might,  for  example, 
have  evaluated  an  option  in  part  on  how  well  the  mission  could  be 
performed  by  the  second-best  choice  of  employment  mechanism  (e.g., 
a  long-range  bomber  from  CONUS,  rather  than  tactical  aircraft  on  a 
combat  vehicle  battle  group  (CYBG)  in  the  region  itself)-  In  principle, 
one  could  attach  “probabilities”  based  on  historical  experience  or  pro¬ 
jections  into  the  future. 

Operational-Employment  Risks.  To  some  extent,  operational  risk 
in  our  example  was  measured  as  1  minus  the  effectiveness  score  (which 
is  something  like  a  probability  of  success  in  such  examples).  However, 
there  is  a  chance  that  we  don’t  understand  what  the  operation  will 
entail  and  that  something  not  considered  in  the  nominal  analysis  will 
bite.  For  example,  surface-to-air  missiles  (SAMs)  might  have  capabili¬ 
ties  that  have  never  been  observed  because  of  strict  operational  secu¬ 
rity;  U.S.  military  headquarters  might  be  infiltrated  by  agents  who 
could  provide  warning  of  operations  and  their  nature;  or  U.S.  military 
networks  might  be  penetrated.  Such  risks  apply  to  all  options,  but  in 
different  ways  and  to  different  degrees.  Experience  would  tell  us,  for 
example,  that  many  of  our  weapon-system  platforms  have  reliabilities 
that  are  well  understood.  In  contrast,  major  surprises  have  historically 
occurred  with  respect  to  the  effectiveness  of  C4ISR  and  sometimes 
with  respect  to  the  ability  to  suppress  air  defenses.  Further,  deception 
practices  can  obfuscate  which  targets  are  real  and  lucrative  and  even 
where  the  correct  aimpoints  actually  are. 

Other  Risks  Associated  with  Operational  and  Strategic  Effects. 
Assuming  that  kinetic  effects  are  achieved  as  intended  and  that  the 
targets  are  destroyed,  how  likely  is  it  that  the  consequences — effects  of 
operational-level  and  strategic-level  effects,  will  be  bad  rather  than 
good?  This  is  significant  for  Global  Strike  options.  A  strike  intended 
to  deter  or  coerce  might  be  misinterpreted  as  the  beginning  of  a  major 
strategic  attack,  precipitating  unfortunate  responses  against  the  U.S. 
allies,  U.S.  forces,  or  even  the  United  States  itself.  Even  if  it  did  not,  the 
effect  might  be  to  firm  up  the  adversary’s  resolve  and  hostility,  rather 


A  Notional  Example:  Global  Strike  97 


than  to  favorably  affect  behavior.  Strategic  bombings  in  the  course  of 
history  have  frequently  had  such  effects.60  Yet  another  possibility  is  that 
a  U.S.  strike  would  be  perceived  unfavorably  by  the  world  generally, 
and  even  by  important  allies.  This  could  be  the  result  of  misunder¬ 
standing  or  simply  of  the  adversary  being  more  successful  than  the 
United  States  in  publicizing  its  narrative,  even  if  that  narrative  were 
false.  To  this  day,  many  people  in  the  Middle  East  continue  to  believe 
that  Israel  was  somehow  responsible  for  the  September  11  attack  on 
the  World  Trade  Center.  During  the  1990s,  many  people  in  Africa 
believed  that  the  CIA  was  somehow  responsible  for  starting  the  HIV 
epidemic.  Many  people  in  China,  including  officials,  believe  that  the 
U.S.  bombing  of  the  Chinese  embassy  during  the  Balkans  conflict  was 
deliberate. 

The  overall  class  of  risks  related  to  effects  should  be  a  matter  of 
great  concern  and  careful  analysis  in  so-called  effects-based  operations. 
Unfortunately,  uncertainty  and  related  risks  are  sometimes  given  short 
shrift.  The  analytic  community  itself  is  sometimes  part  of  the  problem 
here,  because  its  emphasis  on  modern  tools  and  visual  displays  conveys 
a  false  sense  of  certainty  or,  more  subtly,  provides  misleadingly  precise 
estimates  of  uncertainty.61 

Aggregation  of  Risks 

Given  this  lengthy  list  of  risks,  which  we  are  sure  is  incomplete,  how 
does  one  “roll  them  up”  to  summarize  risk  level  succinctly?  As  with  all 
aggregation  challenges  in  portfolio  analysis,  there  is  no  single  answer. 
The  use  of  linear  weighted  sums  is  often  insufficient,  because  in  such 
calculations  the  individual  risks  must  all  be  kept  modest.  Therefore,  we 
have  typically  aggregated  by  having  the  higher-level  summary  risk  be 
evaluated  as  the  worst  of  the  subordinate  risks. 

With  this  background,  Figure  5.17  shows  the  risk  hierarchy  we 
used  in  our  Global  Strike  application.  As  a  practical  matter,  the  lowest- 
level  inputs  were  entered  subjectively  for  each  investment  option,  e.g., 
assuming  that  the  technical-programmatic  risk  for  the  advanced- 
sensors  (ATR/SAR)  option  would  be  moderate  for  the  mobile- 
missile  scenario  but  small  for  the  other  scenarios.  That  is,  the  technical- 
programmatic  risk  would  be  the  same,  independent  of  scenario  of 
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employment,  but  these  capabilities  have  little  effect  on  the  terrorist  or 
WMD-facility  scenarios  and  therefore  contribute  minimal  risk.62 


Illustrative  Conclusions  for  Global  Strike 

Our  summary  conclusions  for  Global  Strike  fall  into  two  categories: 
(1)  conclusions  reached  from  a  cost-effectiveness  perspective,  and  (2) 
conclusions  reached  from  a  perspective  of  maximizing  flexibility,  adap¬ 
tiveness,  and  robustness. 

Investment-Related  Conclusions:  Cost  and  Effectiveness 

Despite  our  emphasis  on  the  notional  character  of  our  applications,  it 
is  useful  to  draw  some  illustrative  conclusions  from  the  analysis.  The 
only  ones  we  shall  mention,  even  as  illustrations,  are  those  that  appear 
to  hold  up  despite  the  manifold  uncertainties  in  data — the  ones  that 
seem  to  fall  out  from  even  first-order  analysis. 

We  present  the  conclusions  in  three  tables:  First,  we  observe 
insights  about  the  challenges;  second,  we  offer  insights  about  solutions; 
and  third,  we  offer  insights  about  cost-effectiveness.  In  each  table,  we 
include  a  column  for  key  assumptions.  From  a  methodological  per¬ 
spective,  this  seems  crucial:  Decisionmakers  don’t  want  lengthy  expla¬ 
nations  about  assumptions,  but  they  do  want  and  need  succinct  flags. 

Table  5.10  presents  three  major  observations  about  the  challenges: 
First,  the  projected  level  of  capability  (with  no  investments  beyond  the 
baseline  program)  is  bleak.  Capability  against  mobile  targets  will  be 
limited  by  ability  to  penetrate  and  ability  to  find  targets;  capability 
against  fleeting  targets,  such  as  terrorist  leaders,  will  be  limited  by  gen¬ 
eral  intelligence  (not  treated  in  this  monograph)  and,  in  some  cases,  by 
execution  time;  capability  against  WMD  facilities  will  be  limited  by 
ability  to  penetrate  to  the  targets,  to  characterize  them  in  enough  detail 
for  precision  attack,  and  to  destroy  them  with  conventional  weapons. 

Table  5.11  makes  some  observations  about  the  potential  solu¬ 
tions  that  have  been  considered.  Of  the  options  considered  for  attack¬ 
ing  mobile  targets,  none  can  provide  high-confidence  future  capability, 
because  at-the-time  capability  will  depend  sensitively  on  counter- 
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Table  5.10 
Insights 


Assertion 


Basis 


Assumptions 


Baseline  projected 
capabilities  are 
inadequate  for  the 
missions  examined  (e.g., 
for  5  to  10  years  from 
today). 

A  niche  problem  is  the 
occasional  need  for  a  very 
short  execution  time  (e.g., 
one  hour). 


A  niche  problem  is  the 
potential  for  deep 
underground  targets 
with  unknown  character 
and  structure. 


Advanced  SAMs,  deeply 
buried  targets,  targets 
with  brief  vulnerability 
times. 


Such  times  may  be 
representative  of  terrorist 
targets,  an  illegal  WMD 
shipment,  or  preparation 
for  a  missile  or  ASAT 
(antisatellite  weapon) 
attack. 

WMD  facilities,  even  Soviet- 
era  leadership  facilities. 


•  No  fixes  are  possible  with 
the  baseline  program 
alone;  new  development 
is  needed. 


•  Execution  time  will 
sometimes  be  the  limiting 
factor  in  a  global  strike. 


•  The  U.S.  might  be  willing 
to  attack  such  targets 
conventionally. 


measures,  counter-countermeasures,  and  associated  tactics.  However, 
there  is  the  potential  for  improved  capabilities.  Even  so,  decisionmakers 
should  give  considerable  thought  to  whether  the  scenarios  of  use — very 
limited  conventional  strikes  into  countries  that  may  possess  weapons  of 
mass  destruction  and  the  means  by  which  to  deliver  them — are  plau¬ 
sible.  Arguments  can  be  made  for  both  sides  of  this  issue,  but  merely 
having  a  modicum  of  military-technical  capability  may  not  be  suffi¬ 
cient.  At  the  same  time,  having  no  such  capability  might  be  deemed 
utterly  unacceptable  (as  policymakers  have  concluded  is  the  case  for 
Ballistic  Missile  Defense). 

Finally,  Table  5.12  offers  some  insights  on  cost-effectiveness  issues. 
These  are,  again,  based  on  rough,  unclassified  data.  The  conclusions, 
however,  are  relatively  robust  because  the  cost  differences  among  the 
options  are  huge.  The  principal  caveats  are  shown  in  the  assumptions 
column. 

Note  that  all  of  the  Global  Strike  options  we  considered  are 
“affordable”  by  the  United  States.  Their  costs  are  small  or  moderate  in 
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Table  5.11 
Possible  Solutions 


Solution 


Basis 


Assumptions 


Counter-mobile-target 
options  would  be  low- 
confidence  investments 
but  would  move  in  a 
good  direction  techno¬ 
logically. 

Solutions  for  mobile- 
missile  targets  have  large 
inherent  risks. 

Trident  II  CTM  in  the  near 
term  is  attractive. 


Joint  operations  with  SOF, 
missiles,  and  aircraft  may 
be  important. 


Enemy  tactics  and 
countermeasures  could  be 
a  problem; 

SAR  and  ATR  provide 
capability;  enhancements 
are  possible. 

Deep,  limited  attacks 
against  states  with 
substantial  capabilities. 

Partially  fills  a  capability 
gap;  is  technologically 
feasible  and  very 
inexpensive. 


SOF  has  potential  for 
special  targets,  given 
capability  for  insertion 
and  extraction. 


•  Intelligent  adversary;  no 
technical  breakthroughs; 
no  silver  bullets. 


•  Such  attacks  would  not 
obviously  be  the  "last 
move"  and  could  trigger  a 
large  retaliation. 

•  Even  low-volume  prompt 
attack  would  be  valuable. 

•  Ambiguity  problems  will 
be  resolved  adequately. 

•  Will  not  weaken  strategic 
deterrent. 

•  Plausibility  of  kinetic 
attacks  of  this  sort; 
operational  feasibility. 


comparison  with,  for  example,  the  cost  of  the  ballistic-missile  defense 
system  (BMDS),  aircraft  modernization,  or  adding  additional  ground- 
force  units  to  the  force  structure.  They  are  also  small  in  comparison 
with  the  costs  of  the  war  in  Iraq.  The  issue,  then,  is  not  affordability, 
but  rather  that  of  wise  choices  given  the  national  budget  for  defense. 
This  said,  the  defense  budget  is  likely  to  come  under  great  pressure 
over  the  next  few  years.  At  that  point,  attitudes  about  affordability  will 
change. 

Seeking  Flexibility,  Adaptiveness,  and  Robustness  in  a  Global  Strike 

The  last  of  our  conclusions  relate  to  the  continuing  theme  that  invest¬ 
ments  should  be  chosen  so  as  to  maximize  flexibility,  adaptiveness,  and 
robustness  (FARness).  This  has  considerable  application  for  the  Global 
Strike  problem.  In  particular,  even  our  illustrative  analysis  was  suffi¬ 
cient  to  identify  the  following  ways  to  achieve  FARness: 
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Table  5.12 

Insights  on  Cost  and  Effectiveness 


Conclusion 


Observation  Assumptions 


Even  with  a  small 
investment  budget 
for  Global  Strike,  fund 
Predators  for  SOF,  Trident 
II  CTM,  and  penaids. 


Cost  to  Global  Strike  (and 
overall)  is  low  for  each. 

Trident  TCM  option 
provides  "niche" 
capability  (low  volume, 
but  very  prompt). 

Funding  would  be  sensible 
even  with  more  far- 
reaching  investments 
(e.g.,  for  a  new  bomber  or 
conventional  ICBM). 

Time  scale  could  be  short. 


Predators  for  SOF  have 
many  applications. 
Even  very  small  CTM 
capability  would  be 
useful. 

Penaids  would  work. 


With  a  larger  investment 
budget,  consider 
funding  SAR  satellite 
constellations,  a  next- 
generation  bomber,  and/ 
or  a  new  ballistic  missile 
for  conventional  strike 
(e.g.,  boost-glide  ICBM  or 
SLBM). 


SAR  satellites  and  a  next- 
generation  long-range 
bomber  would  be 
largely  paid  for  by  other 
capability  areas. 

A  new  ballistic  missile 
would  be  expensive  and 
more  exclusively  for 
Global  Strike. 


•  Plausibility  of  related 
missions. 

•  Successful  R&D. 

•  Global  Strike  would  pay 
only  very  small  share  of 
SAR  and  NGLRS  costs 


NOTE:  SAR  =  synthetic-aperture  radar;  TCM  =  Trident  Conventional  Missile;  penaids  = 
penetration  aids;  NGLRS  =  next-generation  long-range  strike. 


•  Assure  multiple  ways  of  performing  potential  missions  to  hedge 
against  maldeployment,  simultaneous  crises,  the  inability  to  use 
bases,  the  need  for  sequential  attacks,  and  so  on. 

•  Do  not  rely  upon  “special”  capabilities.  Various  types  of  counter¬ 
countermeasures  and  cyber- attack  methods  have  important  poten¬ 
tial  in  some  circumstances,  but  they  may  be  fragile  and  uncertain. 
They  are  therefore  best  not  counted  upon  to  close  shortfalls. 

•  Encourage  competition  among  the  military  Services  and  branches; 
many  ways  exist  to  accomplish  the  various  types  of  global  strike 
that  have  been  identified  by  defense  officials.  More  than  one 
approach  should  be  pursued  in  R&D,  perhaps  through  at  least 
prototype  testing. 

•  Prepare  soberly  for  a  continuing  competition  of  measures  and 
countermeasures.  Since  many  Global  Strike  options  can  be  coun¬ 
tered,  next-round  capabilities  will  be  needed.  This  is  an  eternal 
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struggle  in  warfare,  but  it  is  best  conducted  by  anticipation  and 
investment  rather  than  an  attitude  of  “Well,  first  let’s  develop  a 
capability,  and  then — when  we  see  problems  arising — we  will 
think  about  next  steps.”  The  problem  with  such  an  attitude  for 
the  Global  Strike  domain  is  that  at  least  some  countermeasures 
should  be  expected  sooner  rather  than  later.  There  may  not  be  a 
lengthy  period  of  one-sided  advantage. 

•  Lay  the  base,  with  R&D,  for  capabilities  that  may  in  the  future 
be  crucial  but  are  currently  not  recognized  as  such.  A  continuing 
tension  between  the  Defense  Department  and  Congress  involves 
various  special  weapons,  such  as  earth  penetrators.  There  are 
arguments  on  both  sides  of  this  issue,  but  the  world  situation  and 
intentions  of  nations  change,  sometimes  quickly,  whereas  it  takes 
many  years  to  develop  such  specialized  capabilities.  The  necessary 
technology  development  should  be  completed  whether  or  not  the 
weapons  are  expected  to  be  acquired. 

Strengths  and  Weaknesses  of  the  Approach 

We  believe  this  example  has  illustrated  fairly  well  the  nature  of  our 
methodology  and  related  tools.  Strengths  of  the  approach  include 
(1)  widening  the  set  of  options  considered,  (2)  reducing  the  number 
of  such  options  to  consider  to  manageable  proportions,  (3)  compar¬ 
ing  the  options  in  a  portfolio-analysis  framework  that  allows  one  to 
see  the  strengths  and  weaknesses  (including  risks)  of  the  options  in  a 
single  summary  scorecard,  (4)  being  able  to  drill  down  to  see  why  the 
options  are  evaluated  as  they  are,  and  (5)  a  sophisticated  version  of  cost- 
benefit  analysis  that  highlights  the  extent  to  which  the  relative  cost- 
benefit  “goodness”  of  options  depends  on  perspective,  i.e.,  on  the  rela¬ 
tive  weight  given  to  different  classes  of  mission  and  scenario. 

The  methodology  is  not  intended  to  substitute  for  in-depth  sys¬ 
tems  analysis  and  systems  engineering;  instead,  it  draws  upon  such 
work  in  its  effort  to  provide  a  high-level  top-down  summary  of  the 
options  and  their  relative  merits.  We  might  also  note  that  methodology 
is  only  methodology;  it  does  not  by  itself  have  content.  The  methodol¬ 
ogy’s  virtues  would  be  entirely  undercut  if  it  is  used  poorly — e.g.,  by 
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not  considering  stressful  scenarios,  or  by  considering  only  options  with 
prior  organizational  or  political  support.  As  always,  quality  depends  on 
the  analysts  themselves  (and  the  organizations  in  which  they  work). 


CHAPTER  SIX 


A  Second  Example:  Ballistic  Missile  Defense 


In  this  chapter,  we  sketch  a  second  application  of  the  methodology, 
one  for  the  Missile  Defense  Agency  (MDA).63  This  application  illus¬ 
trates  choosing  among  investments  in  diverse  technologies  and  systems 
for  a  single  mission,  whereas  the  Global  Strike  example  in  Chapter 
Five  involved  investing  with  alternative  missions  in  mind.  Classifica¬ 
tion  guidelines  prohibit  much  discussion  of  even  hypothetical  systems 
and  their  potential  shortcomings  and  vulnerabilities,  but  we  can  make 
a  number  of  methodological  points.  Consistent  with  our  general  meth¬ 
odology  (Figure  3.1),  we  discuss,  in  turn,  defining  the  mission;  devel¬ 
oping  an  appropriate  scenario  space  and  a  parameterized  spanning  set 
of  scenarios;  defining  CONOPS  and  critical  components;  and  iden¬ 
tifying,  evaluating,  and  comparing  options  in  a  portfolio  framework 
that  includes  benefits,  risks,  and  costs.  The  chapter  ends  by  discuss¬ 
ing  some  principles  of  the  portfolio  analysis,  including  the  principle 
that  the  portfolio  chosen  provides  overall  flexibility,  adaptiveness,  and 
robustness. 


Defining  Ballistic  Missile  Defense  Missions 

MDA’s  capability  area  is  Ballistic  Missile  Defense  (BMD).  It  excludes 
air  defense,  including  defense  against  cruise  missiles.  If  a  capability- 
area  review  were  to  approach  the  matter  freshly,  it  might  or  might 
not  conclude  that  the  capability  would  be  extended  to  include  cruise 
missiles.  The  argument  for  doing  so  would  include  the  desire  to 
have  a  comprehensive  view  of  missile-defense  capabilities.  The  argu- 
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ment  for  not  doing  so  would  include  the  recognition  that  defend¬ 
ing  against  air-breathing  missiles  is  different  technically  and  involves 
entirely  different  platforms  and  organizations.  In  business  terms,  one 
might  ask  whether  it  would  be  appropriate  for  MDA  to  go  beyond 
its  “core  competence”  by  getting  into  defense  against  cruise  missiles. 

The  most  important  part  of  MDA’s  mission  statement*  is  that  it 
explicitly  identifies  three  missions:64 

1 .  Homeland  defense 

2.  Defense  of  U.S.  friends  and  allies  (DOFA) 

3.  Defense  of  U.S.  military  forces  deployed  abroad. 

This  is  in  contrast  with  focusing  only  on  homeland  defense  and 
is  consistent  with  national  strategy,  which  recognizes  the  importance 
of  assuring  friends  and  allies  in  various  ways,  of  dissuading  adversaries 
from  aggressive  courses  of  action  abroad,  and  of  using  U.S.  projection 
forces  in  overseas  theaters  of  operation  where  they  might  be  threatened 
by  missile  attacks. 


A  Parameterized  Scenario  Space  for  BMD 

Dimensions  of  the  Problem 

We  discussed  the  generic  issues  involved  in  developing  an  appropri¬ 
ate  scenario  space  and  a  simplified  spanning  set  of  parameterized  sce¬ 
narios  in  Chapter  Three.  The  specific  issues  in  doing  so  for  BMD  are 
unusual  in  several  respects,  primarily  because  the  BMD  mission  for 
homeland  defense  is  exceedingly  challenging  from  a  technical  perspec¬ 
tive — so  much  so  that  debates  have  gone  on  for  years  as  to  what  level 
of  national  BMD,  if  any,  the  United  States  should  seek.  The  United 
States  currently  has  some  deployed  BMD  capability,  but  robust  levels 
of  homeland-defense  capability  will  take  decades  to  achieve  if  they 


As  of  April  2007,  the  mission  statement  was  online  at  http://www.mda.mil/mdalink/ 
html/aboutus.html.  The  motivations  for  the  director’s  strategic  intent  are  rooted  in  Public 
Law  106-38,  1999,  the  National  Defense  Strategy  of  the  United  States,  and  other  public 
documents. 
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can  be  achieved  at  all.  In  contrast,  much  has  been  accomplished  with 
respect  to  theater  defenses  against  shorter-range  ballistic  missiles,  and 
those  efforts  have  never  been  particularly  controversial. 

Figure  6.1  presents  one  view  of  how  to  think  about  the  problem 
space.  Reducing  the  many  dimensions  of  the  problem  to  just  two,  we 
show  a  region  plot  with  U.S.  objectives  for  the  BMDS  along  the  x- 
axis  and  the  difficulty  of  the  threat  along  the  y-axis.  The  purpose  is  to 
indicate  that  the  various  areas  of  the  region  plot  present  very  different 
challenges.  The  difficulty  of  an  area’s  challenge  is  characterized  by  the 
area’s  color  (with  red  indicating  the  highest  level  of  difficulty  and  light 
green  indicating  the  lowest).  Thus,  if  the  U.S.  objective  is  merely  to 
have  a  good  enough  BMDS  that  it  is  not  “naked”  and  is  able  to  inter¬ 
cept  a  small,  simple  attack,  the  challenge  is  well  within  what  is  possible. 
Against  a  small,  simple  attack,  it  should  also  be  possible  to  have  a  high- 
quality  defense  that  would  in  fact  intercept  attacking  missiles.  Thus, 
the  bottom  row  of  areas  is  shown  as  light  green.  Countermeasures  and 

Figure  6.1 

One  View  of  the  Scenario  Space 
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other  aspects  of  threat  sophistication,  however,  make  even  the  ability  to 
deter  somewhat  uncertain,  so  the  colors  shown  along  the  left  column 
are  yellow  for  more-advanced  threats.  As  we  move  rightward  in  the 
figure,  we  see  being  able  to  substantially  disrupt  an  attack  and  per¬ 
haps  thereby  dissuade  a  would-be  attacker  becomes  difficult  (orange) 
for  more-advanced  threats.  The  challenge  of  defending  quite  success¬ 
fully  and  thereby  denying  the  attacker  any  gain,  however,  appears  to 
us  to  be  beyond  the  pale  for  a  large-scale  attack  by  a  near  peer  (i.e.,  a 
country  with  technical  and  military  prowess  akin  to  that  of  the  former 
Soviet  Union). 

At  this  stage  of  history,  U.S.  BMD  capability  (deter  or  dissuade, 
for  a  simple  rogue)  is  probably  somewhere  in  the  light  green  or  yellow 
areas  at  the  bottom  left  of  Figure  6.1.  As  the  United  States  plans 
improved  capabilities,  whether  or  not  they  are  physically  deployed, 
what  cases  should  it  write  into  its  planning  scenarios?  Our  own  view  is 
that,  at  present,  the  three  points  shown  represent  an  adequate  spanning 
set  from  the  perspective  of  this  simple  representation.  It  is  reasonable 
to  measure  future  BMDS  options  by  the  criteria  associated  with  those 
points.  Presumably,  U.S.  objectives  against  a  simple  rogue,  or  even  a 
relatively  advanced  rogue,  should  be  stringent.  At  the  same  time,  it 
would  be  wasted  effort — for  now — to  focus  analysis  on  the  top  right- 
hand  corner.  That  is  not  currently  an  area  of  interest,  and  the  systems 
currently  programmed  or  under  serious  consideration  would  not  deal 
with  it. 

So  far,  we  have  discussed  only  a  very  simple  conception  of  the  sce¬ 
nario  space.  Table  6.1  shows  what  we  see  as  the  principal  dimensions 
of  the  more  technically  meaningful  scenario  space,  along  with  what  we 
regard  as  natural  measures  of  effectiveness. 

We  can  elaborate  somewhat  on  the  dimensions,  as  follows: 

1 .  Planning  for  BMDS  must  consider  the  near  to  mid-term  because 
problems  exist  in  today’s  world,  and  more  serious  problems 
could  arise  quickly.  However,  advanced  BMD  capabilities  will 
take  decades  to  develop.  When  evaluating  alternative  programs, 
all  of  these  timescales  matter. 
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Table  6.1 

Parameterizing  Scenario  Space  and  Characterizing  Capability 


Dimensions 


Measures  of  Effectiveness 


•  Time  frame:  near  to  mid-,  long,  and  very  •  Probability  of  intercept  (or  its  inverse, 
long  term  probability  of  target  penetration) 


Geography:  target  and  launch  locations 
Size  of  attack 

Attack  tactics  (e.g.,  sequential  or  salvo, 
one-time  or  continuing) 


Confidence  in  U.S.  probability  of 
intercept 

Expected  number  of  leakers 
Nominal  probability  of  zero  leakers 


•  Countermeasure  cases 


•  Probability  of  systematic  failure 


•  Strategic  warning  needed 

•  Size  and  configuration  of  the  BMDS 


•  Risk 


2.  The  primary  geographic  issues  are  the  range  of  the  attack,  which 
relates  to  whether  defenses  must  cope  with  something  like  a 
short-range  Scud  missile  or  an  ICBM,  and  whether  the  United 
States  has  forces  in  the  right  positions  to  intercept  an  attack¬ 
ing  missile.  Particularly  important  is  whether  the  United  States 
would  be  able  to  intercept  a  missile  in  its  boost  phase,  during 
which  it  is  potentially  quite  vulnerable  and  is  a  single  target.65 

3.  Size  of  the  attack  is  an  important  variable  because  some  defense 
systems  can  handle  larger  attacks  better  than  others  can.  Sim¬ 
ilarly,  some  defense  systems  are  far  more  capable  than  others 
against  tactically  complex  attacks. 

4.  Next,  there  is  the  matter  of  countermeasures.  The  voluminous 
public  literature  on  countermeasures  to  BMD  systems  dates 
back  to  the  1950s  (Carter  and  Schwartz,  1984;  Office  of  Tech¬ 
nology  Assessment,  1986,  2002),  although  it  would  be  a  mis¬ 
take  to  depend  on  that  literature  exclusively,  because  it  tends  to 
be  written  by  BMD  skeptics  and  because  MDA  is  pursuing  a 
variety  of  counter-countermeasures  that  are  not  discussed  in  the 
open  literature.  For  our  purposes,  it  is  sufficient  to  assert  that 
there  are  countermeasure  classes  and  to  denote  them  as,  A,  B, 
....  With  this  simplification,  BMD  analysis  should  separately 
characterize  capability  against  threats,  using  the  various  classes 
of  countermeasure. 
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5.  Dependence  on  strategic  warning  is  an  issue  because  some  com¬ 
ponents  of  the  BMDS,  such  as  ships  and  sea-based  radars  or 
airborne  lasers,  may  or  may  not  be  deployed  appropriately  at  the 
time  of  a  crisis.  Other  components,  such  as  long-range  ground- 
based  interceptors,  are  less  sensitive  to  this  issue. 

6.  Obviously,  effectiveness  will  also  depend  on  the  size  of  the 
defense  system,  where  its  components  are  deployed,  and  so  on. 

As  for  measures  of  effectiveness,  simplicity  has  considerable  merit. 
From  an  analytical  viewpoint,  the  best  single  measure  is  the  fraction 
of  an  attacking  force  that  would  be  intercepted.  However,  the  conse¬ 
quences  of  failing  to  intercept  an  attacker  would  be  quite  severe,  so  it 
is  also  important  to  measure  the  confidence  of  the  assessment.  If  the 
expected  result  would  be  interception  of  90  percent  of  the  attacking 
missiles,  how  likely  is  it  that  the  result  would  instead  be  50  percent 
or  100  percent?  The  methods  of  calculation  involve  some  statistical 
uncertainties,  but — far  more  important — there  are  complications  such 
as  might  be  encapsulated  in  the  statement,  “Well,  Mr.  President,  we 
should  intercept  virtually  all  of  the  attackers  if  we  understand  the  ene¬ 
my’s  systems  correctly.  If  we  don’t,  and  if  he  has  a  countermeasure  for 
which  we  are  unprepared,  then  we  might  not  intercept  anything.  That’s 
unlikely,  however  (perhaps  one  chance  in  ten),  so  let’s  report  90  percent 
confidence.”  Clearly,  the  meaning  of  “90  percent”  here  is  different  than 
it  would  be  if  one  said,  “Well,  Mr.  President,  there  are  random  failures 
here  and  there,  so  we  should  expect  to  intercept  only  90  percent  of  the 
attack.” 

When  dealing  with  small  attacks  involving  nuclear  weapons, 
many  analysts  have  discovered  over  the  years  that  policymakers  want 
to  know  the  probability  that  all  of  the  attack  can  be  stopped.  That  is 
what  we  mean  by  the  nominal  probability  of  zero  leakers.  It  is  a  misno¬ 
mer,  however,  and  potentially  very  misleading,  because  of  the  problem 
mentioned  above:  It  measures  statistical  confidence,  but  only  for  the 
case  in  which  the  attacking  system  and  the  defense  system  are  ade¬ 
quately  understood.  Systematic  errors  are  a  special  issue.  A  key  issue  in 
any  systems-engineering  analysis  is  assuring  the  absence  of  common¬ 
mode  failures  that  obviate  the  benefits  of  firing  multiple  interceptors  at 
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a  given  target,  or  having  what  are  intended  to  be  independent  defense 
layers  for  boost,  midcourse,  and  terminal  phases  of  flight.  Some  exam¬ 
ples  might  be  a  critical  flaw  in  the  global  command  and  control  system 
or  a  critical  flaw  in  the  kill  device  used  on  several  types  of  interceptors. 
Despite  uncertainties  about  such  matters,  analysis  can  represent  the 
implications  of  degrees  of  containment.  Our  colleague  David  Vaughan 
has  done  mathematical  work  on  such  issues  in  the  past,  and  we  have 
incorporated  related  algorithms  in  RAND’s  capabilities  analysis  model 
for  missile  defense  (CAMMD)  (Willis,  Bonomo,  Davis,  and  Hillestad, 
2006). 

A  Perspective  on  Capabilities-Based  Planning 

Because  developing  and  fielding  a  BMDS  is  so  challenging  and  the 
baseline  as  of  a  few  years  ago  was  so  poor,  commanders  at  MDA  have 
made  a  clear  distinction  between  what  capabilities  they  will  be  able 
to  deliver  at  a  point  in  the  future  and  whether  those  capabilities  are 
sufficiently  good  to  merit  deployment.66  As  indicated  in  Figure  6.2, 
the  expectation  has  been  to  develop  systems  with  increasing  degrees  of 
capability  over  time.  Figure  6.2  is  merely  a  cartoon,  but  it  suggests  an 
image  of  starting  with  minimal  capability  and  slowly  moving  outward, 
with  emphasis  on  solving  the  problems  posed  by  geography,  counter¬ 
measures,  the  need  for  warning,  and  attack  tactics,  but  not  the  problem 
of  attack  size.  It  further  acknowledges  that  the  problems  of  counter¬ 
measures  are  unlikely  to  be  fully  solved.  The  path  the  United  States 
actually  follows  remains  to  be  seen. 


CONOPS  and  Critical  Components 

For  Ballistic  Missile  Defense,  a  given  concept  of  operations  (CONOPS) 
is  often  described  by  reference  to  an  Engagement  Sequence  Group 
(ESC).  Successful  intercept  of  an  attacking  ballistic  missile  requires 
detecting  and  tracking  the  attacker,  launching  and  guiding  an  intercep¬ 
tor  (or  high-powered  laser),  engaging  and  destroying  the  attacker,  and 
assessing  results.  This  may  be  accomplished  in  a  myriad  of  ways,  draw¬ 
ing  upon  whatever  component  systems  are  available  and  appropriate 
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Figure  6.2 

Capabilities  Versus  Time 


for  the  particular  engagement.  The  components  can  include  air-,  sea-, 
or  space-based  sensors  and  interceptors.  A  centralized  command  and 
control  system  is  involved,  but  so  also  are  more-regional  systems.  Battle 
management  may  be  quite  complex  because  of  the  broad  stretch  of 
geography  involved  and  also  the  need  to  cross  command  and  control 
boundaries.  That  is,  an  attack  may  be  launched  in  one  combatant  com¬ 
mander’s  (COCOM’s)  area  of  responsibility,  pass  through  another’s, 
and  then  approach  the  United  States  itself,  which  is  NORTHCOM’s 
responsibility.  U.S.  STRATCOM  has  the  overall  lead  for  strategic 
defense,  but  battle  management  must  be  globally  integrated. 

Defining  a  given  CONOPS  for  even  high-level  systems  engineer¬ 
ing  requires  a  finer  level  of  detail  because,  for  example,  a  sensor  may 
be  adequate  for  approximate  tracking,  but  other  sensors  may  be  needed 
for  fire  control;  multiple  sensors  may  be  needed  for  discrimination  or 
traffic  management;  and  interceptors  and  their  kill  vehicles  need  the 
capability  to  do  final  discrimination,  maneuver,  and  kill  of  separate 
targets. 
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Analysis  of  BMD  capability,  then,  requires  paying  attention  to 
whether  all  of  the  necessary  elements  are  present,  both  at  the  level  of  a 
viable  ESG  and  at  finer  levels  within  it.  We  shall  not  discuss  the  BMD- 
specific  matters  further  in  this  monograph,  but  the  issue  of  identifying 
critical  components  and  doing  capability  analysis  with  the  hard-nosed 
view  that  all  critical  components  must  work  is  a  fundamental  part  of 
the  overall  methodology,  as  discussed  in  Chapter  Three. 


Framework  for  Portfolio  Analysis 

Top-Level  Structure 

A  basic  problem  in  portfolio  analysis,  which  we  have  encountered  every 
time  we  have  applied  it,  is  that  the  number  of  dimensions  exceeds  what 
can  reasonably  be  displayed  to  decisionmakers  in  a  summary  account. 
It  comes  down  to  the  number  of  columns  that  can  be  shown  without 
causing  eyes  to  glaze.  This  is  a  problem  familiar  to  analysts,  and  there 
is  no  ultimate  solution.  Table  6.2  shows  a  simplified  depiction  of  a  pos¬ 
sible  summary  for  the  BMD  problem,  for  a  context  in  which  decision- 

Table  6.2 

Notional  Top-Level  Portfolio  Structure  for  BMD  Analysis 


Option 

Defend 

homeland 

Defend 
allies,  friends, 
deployed  forces 

Strategic 

risk 

Technical 

and 

schedule 

risk 

Annualized 
20-year  cost 
(relative) 

Near 

term 

Long 

term 

Near 

term 

Long 

term 

A 

Y 

R 

Y 

Y 

R 

LG 

1.0 

B 

LY 

R 

LY 

LG 

0 

LG 

1.2 

C 

LY 

LY 

LY 

LY 

LY 

LY 

1.5 

D 

LY 

LG 

LY 

LY 

LG 

0 

2.0 

E 

LY 

LG 

LY 

LG 

G 

2.2 

NOTE:  Wholly  notional  options,  evaluations,  and  costs. 
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makers  are  considering  alternative  missile-defense  programs  and  view¬ 
ing  effectiveness,  cost,  and  risk  simultaneously. 

Table  6.2  condenses  the  top-level  summary  into  a  chunk  for 
homeland  defense  and  a  combined  chunk  for  defending  both  allies  and 
friends  and  deployed  U.S.  forces.  It  then  has  a  column  for  strategic  risk, 
a  column  for  technical  and  schedule  risk,  and  a  column  for  relative 
cost.  The  second  through  fourth  columns  show  estimated  effective¬ 
ness  in  defense.  The  estimates  are  an  aggregation  of  results  for  various 
scenarios  and  cases.  Strategic  risk  is  a  partially  subjective  assessment  of 
the  degree  to  which  a  given  program  would  prepare  the  United  States 
for  anticipating  or  responding  very  quickly  to  serious  threats  involving 
ballistic  missiles.  Technical  and  schedule  risk  is  a  combined  and  par¬ 
tially  subjective  assessment  of  the  degree  of  risk  involved  in  achieving 
the  level  of  effectiveness  asserted  for  the  option. 

Although  the  options  are  unspecified  and  the  evaluations  and 
costs  are  purely  notional,  the  story  being  told  is  that  the  baseline  option 
(option  A)  will  lead  to  substantial  strategic  risk.  It  will  not  be  very  trou¬ 
blesome  technically  or  programmatically,  and  won’t  cost  very  much, 
but  the  results  will  be  mediocre  and  the  strategic  risk  high.  Option  B 
would  spend  more  to  shore  up  BMD  capability  in  important  theaters  of 
operation  such  as  East  Asia  or  the  Middle  East.  It  would  do  little  about 
homeland  defense,  however,  so  risks  would  remain  high — a  bit  lower 
than  in  option  A  because  of  the  theater  defenses,  but  still  worrisome. 

Option  C  would  invest  more  in  systems  for  homeland  defense, 
which  would  somewhat  improve  projected  capability  and  would  lay 
the  base  for  greater  effort  if  that  should  be  necessary.  Thus,  it  would 
reduce  strategic  risk  but  increase  technical  risk,  because  the  systems  at 
issue  require  the  success  of  new  programs.  Option  D  is  an  even  more 
ambitious  investment  in  homeland  defense,  one  that  might  deliver  rela¬ 
tively  substantial  capability  in  ten  years  or  so.  The  technical  risk,  how¬ 
ever,  would  be  even  greater  for  the  systems  at  issue.  Finally,  option  E  is 
something  approaching  a  crash  across-the-board  effort.  In  this  notional 
analysis,  it  would  deliver  substantial  capabilities  for  both  theater  and 
homeland  defense  and  would  greatly  reduce  strategic  risk  (associated 
with  ballistic  missiles).  The  technical  risks  would  remain  high,  as  for 
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option  D,  and  in  addition,  the  crash  program  would  entail  major  cost- 
schedule  risks. 

The  rightmost  column  suggests  that  the  ratio  of  costs  varies  from 
a  baseline  value  to  something  more  than  twice  as  great.  Since  the  BMD 
program  cost  is  roughly  $10  billion/year,  the  most  expensive  program 
might  assume  the  expenditure  of  another  $12  billion/year  over  the  next 
20  years. 

Note  that  the  display  is  not  the  classic  presentation  of  three 
options,  the  best  of  which  is  always  the  middle  one.  Instead,  it  conveys 
a  sense  of  choice.  If  decisionmakers  were  convinced  that  theater  BMD 
was  a  serious  problem  that  could  be  greatly  mitigated,  whereas  home¬ 
land  defense  was  technically  dubious,  expensive,  and  apt  to  worsen 
international  relations,  they  might  choose  option  B.  In  contrast,  if 
they  concluded  that  homeland  defense  was  an  urgent  priority  and  that 
the  technology  was  within  grasp  (albeit  risky),  they  might  tilt  toward 
option  D  or  E. 

Zooming  for  Explanation  of  Summary-Level  Results 

Although  we  do  not  describe  the  process  further  in  this  monograph, 
in  our  work  for  MDA,  our  summary-level  results  (in  something  analo¬ 
gous  to  Table  6.2)  were  undergirded  by  work  with  a  capabilities  model 
(CAMMD)  and  extensive  interactions  with  MDA’s  systems  engineers. 
These  technical  and  personal  interactions  were  crucial,  because  so 
much  of  what  matters  in  BMD  analysis  can  be  understood  only  with  a 
relatively  deep  understanding  of  the  relevant  physics,  engineering,  and 
even  operations.* 

Zooms  to  explain  a  summary  chart  such  as  the  notional  Table  6.2 
would  address  capability  against  a  variety  of  adversaries  with  a  variety 
of  countermeasures. 


The  RAND  team,  led  by  James  Bonomo  and  Paul  Davis,  benefited  from  work  with  Dr. 
Edward  Gerry  and  other  members  of  MDA’s  National  Team,  which  draws  from  industry, 
FFRDCs,  and  national  laboratories.  It  was  gratifying  to  learn  that  we  could  use  our  top- 
down  portfolio-analysis  tool  and  our  high-level  capability  model  to  connect  with  and  sum¬ 
marize  from  the  results  of  systems-engineering  work. 
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Shortfalls,  Opportunities,  and  Surpluses  for  the  BMD 
Program 

It  is  not  possible,  for  reasons  of  classification,  to  characterize  the  cur¬ 
rent  or  projected  effectiveness  of  the  BMDS  in  a  meaningful  way. 
However,  MDA’s  Director,  Lt.  General  Henry  (Trey)  Obering,  has  tes¬ 
tified  regularly  before  Congress  and  has  sought  to  “underpromise  and 
overdeliver.”  In  a  summary  comment  at  a  news  conference,  in  response 
to  questions  about  capability,  he  said* 

Well,  it  depends  on  .  .  .  what  type  of  countermeasures  that  the 
North  Koreans  or  other  nations  could  use.  ...  I  feel  confident 
we  can  handle  simple  countermeasures,  just  like  I  said  ...  I  was 
confident  in  the  overall  performance  of  the  system.  We  will  prove 
that  in  the  tests  to  come.  But  also,  one  of  the  reasons  I  say  that  is 
that,  as  I  said,  we  have  flown  countermeasures  against  this  system 
in  the  past.  We  had  a  prototype  of  the  kill  vehicle  that  we  used 
today,  and  we  used  countermeasures  against  that  prototype.  And 
so  the  ability  to  discriminate  between  those  countermeasures  and 
the  RV  [reentry  vehicle]  has  been  demonstrated  in  the  testing  that 
we  did  in  the  2001-2002  time  frame. 

So  that’s  why  we  test  to  where  we  have — we  believe  we  have  some 
capability,  we  put  it  in  the  field.  We  test  some  more  systematically, 
one  step  at  a  time,  put  some  more  capability  into  the  system.  So 
we’re  trying  to  do  this  in  an  evolutionary  fashion  where  we  grow 
the  capability,  because  we  want  to  make  sure  that  we’re  doing  it 
the  right  way. 


Identifying,  Evaluating,  and  Comparing  BMD  Options 

The  building-block  options  that  have  been  considered  for  the  BMDS 
include  those  in  Table  6.3  (and  some  others  that  we  do  not  consider  in 


From  DoD  News  Transcript,  September  1,  2006  (http://www.defenselink.mil/Transcripts/ 
Transcript. aspx?TranscriptID=3710). 
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Table  6.3 

Building-Block  Options  for  Ballistic  Missile  Defense 


Component 

Phase  for 

Intercept  of  Target 

Timescale 

Comments 

Aegis  ships  and  SM-3 

Ascent,  midcourse 

Near  term 

interceptors 

Patriot  PAC-3 

Terminal 

Near  term 

THAAD 

Terminal 

Mid-term 

Ground-Based  Midcourse 
Defense  (GMDS) 

Midcourse 

Near  term 

Based  in  Alaska 
and  Vandenberg 

Kinetic-energy  interceptor  (KEI) 

Boost,  midcourse 

Long  term 

Multiple-kill  vehicle  (MKV) 

Midcourse 

Long  term 

Space-based  surveillance  and 

Boost,  midcourse 

Mid-term 

tracking  system  (STSS) 

Sea-based  X-band  radar  (SBX) 

Forward-deployed  radars 

Boost,  midcourse 

Near  to  mid¬ 
term 

Airborne  laser  (ABL) 

Boost 

Mid-term 

Advanced- Discrimination 

Midcourse 

Mid-term 

Initiative  (ADI) 

Air-launched  interceptor 

Boost,  midcourse 

Not  currently 
being  funded 

Space-based  interceptors  (SBIs) 

Boost,  midcourse 

Long  term 

Not  currently 
being  funded 

Space-based  lasers  (SBLs) 

Boost,  midcourse 

Very  long 
term 

Not  currently 
being  funded 

NOTE:  Elements  of  the  BMD  program  are  described  on  the  MDA  web  site;  in  particular, 
see  http://www.mda.mil/mdalink/html/factsheet.html;  http://www.mda.mil/mdalink/ 
html/asptinitiative.html;http://www.mda.mil/mdalink/html/asptmkv.html;  and  http:// 
www.mda.mil/mdalink/html/japan.html. 

this  discussion).  All  of  these  are  noted  on  the  MDA  web  site.*  Impor¬ 
tantly,  however,  many  of  the  components  in  Table  6.3  are  already 
in  the  baseline  MDA  program.  The  typical  decision  issues  concern 
whether  to  increase  or  decrease  funding  for  them.  Thus,  a  given  deci¬ 
sion  option  might  be  about  whether  to  add  additional  ground-based 
missile-defense  batteries,  to  accelerate  the  Kill-Vehicle  program,  or  to 
slow  one  or  another  development. 


For  fact  sheets  on  these  items,  see  http://www.mda.mil/mdalink/html/factsheet.html. 
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We  have  not  as  yet  applied  our  option-generation-and-screen- 
ing  methodology  to  Ballistic  Missile  Defense.  In  principle,  however, 
doing  so  should  be  a  useful  exercise — especially  if  some  of  the  building 
blocks  involve  a  reduced  level  of  effort  in  some  initiatives.  Some  of  the 
military-economic  tradeoffs  obtained  might  also  be  valuable. 

In  our  work  for  MDA,  we  have  emphasized  some  particular  issues 
in  evaluating  options.  With  the  encouragement  of  senior  MDA  offi¬ 
cials,  we  have  sought  to 

•  Highlight  shortcomings,  rather  than  performance  only,  for  stan¬ 
dard  threat  cases.  That  is,  decisionmakers  should  be  able  to  see 
quickly  what  advances  will  be  made  by  the  option  at  issue  and 
what  shortcomings  will  remain. 

•  Clearly  characterize  expected  performance  against  adversaries 
exhibiting  a  range  of  countermeasures. 

•  Characterize  the  program’s  balance  across  MDA’s  missions  of 
homeland  defense,  defense  of  friends  and  allies,  and  defense  of 
U.S.  forward-deployed  forces. 

•  Characterize  the  relative  technical  risk  of  options,  thereby 
establishing  values  for  hedging,  as  with  overlapping  technical 
approaches,  competition  among  contractors  or  military  Services, 
and  so  on. 

•  Characterize  the  degree  to  which  an  option  establishes  the  R&D 
base  for  later  developments  that  may  be  necessary  even  though 
they  are  not  currently  being  emphasized  in  U.S.  policy  and  con¬ 
gressional  funding. 

These  actions  may  seem  obvious  to  a  reader  not  involved  in  BMD 
issues,  but  none  of  them  are  so  obvious  in  practice.  For  example,  it  is 
natural  for  an  organization  to  develop  summary  presentations  that  put 
its  preferred  options  in  a  good  light — highlighting  the  expected  accom¬ 
plishments  but  downplaying  continued  shortcomings  that  might  be 
criticized  by  outside  observers.  This  has  been  especially  true  in  BMD 
work,  because  the  technical  problem  is  so  difficult  and  the  risks  are 
high.  Many  critics  are  against  all  but  a  minimal  BMD  program,  if 
that,  believing  that  the  challenge  is  impossible  and  that  investments 
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made  in  BMD  are  wasted.  As  reflected  in  public  law  in  1999,  however, 
the  United  States  decided  that  it  must  pursue  a  BMDS,  even  one  with 
limited  capability,  because  BMD  capabilities  are  deemed  useful  and 
can  be  seen  as  deliberate  movement  toward  something  more  substan¬ 
tial  over  time.  Decisionmakers  have  also  taken  the  view  that  having 
no  defense  system  is  unacceptable  in  an  era  of  dangerous  states  with 
WMD  and  ballistic-missile  programs. 

The  issue  of  balance  is  important  because,  over  the  decades  (not 
so  much  in  recent  years),  the  national  mood  (along  with  congres¬ 
sional  funding)  has  gyrated  among  objectives:  In  some  years,  the  focus 
has  been  on  homeland  defense;  in  other  years,  it  has  been  on  theater 
defense  (of  both  allies  and  forward-deployed  forces).  In  some  years, 
the  emphasis  has  been  on  achieving  minimal  capability,  while  in  other 
years,  it  has  been  on  the  pursuit  of  substantial  defense  capability.  All  of 
this  is  understandable  in  a  complex  world,  but  major  capability  develop¬ 
ments  do  not  do  well  in  the  midst  of  such  gyrations  in  policy  and  funding. 
In  recent  times,  the  tension  has  been  more  between  preparing  a  techni¬ 
cal  base  for  future  deployment  decisions  and  deploying  currently  avail¬ 
able  systems. 

Another  reason  for  our  emphasis  on  portfolio  balance  is  that, 
from  a  strategic  perspective,  all  of  MDA’s  three  missions  are  and  will 
continue  to  be  important.  It  would  arguably  be  irresponsible  for  the 
United  States  to  pursue  a  BMD  program  that  ignored  some  of  the  mis¬ 
sions  because  of  mood  swings.  To  be  sure,  the  relative  balance  of  fund¬ 
ing  among  the  missions  may  very  well  change  over  time  as  the  result 
of  decisions  by  both  national  leaders  and  the  voting  public,  but  it  is  a 
responsibility  of  planners  to  think  in  terms  of  establishing  the  right 
balance,  rather  than  of  tilting  drastically  in  one  direction  or  another.  It 
is  quite  possible  that  technical  options  forgone  now  will  be  demanded 
later,  with  expressions  of  incredulity  when  it  is  learned  that  they  had 
not  been  pursued.  Let  us  stress,  however,  that  we  are  referring  primar¬ 
ily  to  research  and  development,  perhaps  through  prototyping,  rather 
than  to  deployment.  That  is,  our  recommendations  tend  to  be  toward 
establishing  a  strong  technical  base  of  potential  capability  and  distin¬ 
guishing  that  sharply  from  deployed  capability.  As  the  world  changes, 
national  sentiments  about  the  need  for  deployed  BMD  will  and  should 
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change  as  well.  DoD’s  technical  and  industrial  base  should  be  able  to 
respond  quickly. 

From  a  decision-analytic  perspective,  the  key  points  here  are  the 
following: 

•  Basic  research  is  inexpensive;  development  is  more  expensive; 
development  and  testing  is  more  expensive  yet;  but  deployment 
with  the  associated  operations  and  maintenance  costs  for  an 
indefinite  future  is  very  expensive.  The  ratios  here  are  something 
like  1:5:10:50. 

•  Investments  in  R&D  provide  future  options ,  which  may  or  may 
not  succeed,  and  even  if  they  do,  they  may  or  may  not  be  pursued. 
Successful  options,  however,  are  potentially  quite  valuable. 

•  The  regrets  associated  with  not  having  laid  the  groundwork  in 
R&D  could  be  quite  severe.  Such  R&D  takes  years,  and  even 
longer  if  it  is  begun  without  a  warm  industrial  base,  and  the 
threats  against  which  options  might  be  deployed  can  emerge 
almost  overnight. 

The  careful  reader  of  this  monograph  will  have  noticed  that  the 
philosophy  we  have  expressed  is  consistent  with  the  overall  theme  of 
seeking  investment  strategies  that  maximize  flexibility,  adaptiveness, 
and  robustness,  rather  than  expected  results  for  some  allegedly  best- 
estimate  image  of  the  future  and  conflicts  within  that  future. 


CHAPTER  SEVEN 


Conclusions  and  Next  Steps 


Condusions 

The  primary  conclusions  of  this  monograph  are  the  following: 

•  The  relatively  generic  methodology  we  have  described  and  dem¬ 
onstrated  can  be  used  to  guide  and  report  results  of  defensewide 
capability-area  reviews  such  as  those  now  under  way  in  DoD,  par¬ 
ticularly  the  new  integrative  Concept  Decision  Reviews  that  are  a 
highlight  of  USD  (AT&L)’s  strategy. 

•  The  methodology  can  be  depicted  in  a  simple  process  diagram, 
but  one  that  includes  important  new  analytical  features  that  are 
frequently  not  present  in  reviews. 

•  The  enablers  of  the  methodology  include  underlying  capability 
models  for  the  area  under  study,  a  tool  for  generating  and  screen¬ 
ing  options  to  identify  good  candidates  for  richer  analysis  and 
comparison,  and  a  portfolio-analysis  tool  that  organizes  coherent 
discussions  that  simultaneously  address  effectiveness,  risk,  and 
cost. 

The  methodology  has  been  applied,  to  various  degrees,  over  the 
course  of  the  past  decade.  In  this  monograph,  we  have  summarized  a 
notional  application  to  Global  Strike  and  an  application  to  Ballistic 
Missile  Defense. 

Although  the  methodology  and  related  tools  are  powerful,  they 
are  hardly  panaceas.  The  limiting  factors  in  analysis  will  continue  to  be 
the  analysts  themselves  and  the  organizations  in  which  they  work. 
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Next  Steps 

A  traditional  part  of  a  final  chapter  is  discussion  of  what  should  come 
next.  Several  important  topics  that  have  been  given  short  shrift  so  far 
in  our  work  should  have  priority  attention  in  subsequent  work: 

•  Analysis  cutting  across  capability  areas 

•  Strategic-level  portfolio  analysis 

•  Advancing  the  science  of  exploratory  analysis 

•  Tool  refinement 

Analysis  Cutting  Across  Capability  Areas 

Extending  the  analysis  methodology  to  cut  across  capability  areas  is 
crucial,  since  major  decisions  will  often  depend  on  whether  an  option 
would  provide  benefits  in  several  capability  areas.  Also,  the  cost¬ 
sharing  issue  mentioned  earlier  can  be  addressed  only  this  way.  DoD 
leaders  hope  to  have  those  responsible  for  capability  areas  do  a  great 
deal  of  “portfolio  balancing”  within  their  respective  domains,  finding 
bill-payers  as  well  as  new  contributors.  However,  in  many  cases,  it  is 
necessary  to  do  cross-area  analysis  in  order  to  properly  evaluate  the 
value  of  a  new  system  or  activity,  the  implications  of  cutting  back  on 
another  system  or  activity,  or  the  way  to  think  about  tradeoffs.  This 
work  should  again  be  targeted  at  DoD’s  tri-chair  leaders  of  capability 
development,  i.e.,  the  USD  (AT&L),  VCJS,  and  DPA&E. 

Strategic-Level  Portfolio  Analysis 

Although  the  work  we  have  reported  here  was  accomplished  for  the 
USD  (AT&L)  and  his  office,  with  others  in  mind,  such  as  the  DPA&E 
and  the  VCJS,  the  same  methods  can  and  should  be  applied  from  a  top 
level — both  within  DoD  and  across  cabinet  departments.  Such  work 
may  be  of  particular  importance  in  the  near  future  as  the  activity  in 
Iraq  slowly  winds  down  and  policymakers  contemplate  alternative  rela¬ 
tive  emphases  in  defense  and  national  security  for  the  years  ahead — 
e.g.,  the  relative  emphasis  on  the  “long  war,”  various  regional  instabili¬ 
ties,  and  the  long-term  competition  in  and  status  of  East  Asia.  What 
implications  should  there  be  for  relative  investment  in  recapitalizing 
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the  armed  forces  after  some  years  of  intense  activity  that  has  worn  them 
down,  modernization  along  the  lines  associated  with  late-1990s  trans¬ 
formation,  restructuring  for  low-intensity  conflicts  and  operations,  or 
shifts  in  the  pattern  of  military  presence  worldwide?  The  first  versions 
of  the  portfolio-analysis  methods  described  in  this  monograph  were 
actually  developed  for  such  strategic-level  work  in  the  mid-1990s  and 
proved  useful  in  casting  the  debates  of  that  time  in  strategic  terms.*  It 
is  arguably  time  for  the  strategic  level  of  analysis  to  be  revisited.  Such 
work  would  best  be  done  on  a  cross-cutting  basis,  with  the  Office  of 
the  Under  Secretary  of  Defense  for  Policy  as  a  major  client. 

Advancing  the  Science  of  Exploratory  Analysis 

As  noted  throughout  this  monograph,  exploratory  analysis  in  pursuit 
of  investment  strategies  that  are  flexible,  adaptive,  and  robust — rather 
than  optimized  for  a  particular  image  of  the  future — is  essential.  Great 
progress  has  been  made  over  the  past  decade  in  developing  methods 
for  exploratory  analysis  and  applying  them  to  actual  problems.  A  new 
wave  of  challenges,  however,  is  posed  by  portfolio  analysis.  In  portfolio 
work,  the  uncertainties  have  a  different  character,  both  substantively 
and  technically.  Significant  research  will  be  needed  to  extend  the  cur¬ 
rent  theory  and  methods  and  to  make  them  practically  available  to  the 
Department  of  Defense. 

Tool  Refinement 

In  the  course  of  the  work  described  in  this  monograph,  we  made  sub¬ 
stantial  enhancements  to  the  Portfolio-Analysis  Tool  (PAT)  and  devel¬ 
oped  a  new  tool  (BCOT).  These  tools  are  ready  for  use,  as  we  have 
demonstrated  in  our  own  work,  but  further  work  is  needed  on  both, 
especially  efforts  to  refine  them  so  that  they  can  be  more  easily  used, 
understood,  modified,  and  maintained  by  people  other  than  their  orig¬ 
inal  developers. 


The  work  contributed  significantly  to  development  of  the  1997  Quadrennial  Defense 
Review  and  to  subsequent  work,  over  two  administrations,  on  transformation  (Davis  et  al., 
1996;  Davis,  Kugler,  and  Hillestad,  1997;  Davis,  Gompert,  Hillestad,  and  Johnson,  1998; 
Hillestad  and  Davis,  1998;  Davis,  2002). 


APPENDIX  A 


Joint  Capability  Areas 


The  current  Joint  Capability  Areas  (JCAs)  are  listed  in  Table  A.l.  How¬ 
ever,  the  list  is  subject  to  change  from  time  to  time.* 

Table  A.l 

Joint  Capability  Areas 

Battlespace  Awareness 
Command  and  Control 
Network  Operations 
Interagency  Coordination 
Public  Affairs  Operations 
Information  Operations 
Protection 
Logistics 

Force  Generation 
Force  Management 
Homeland  Defense 
Strategic  Deterrence 
Shaping  &  Security  Cooperation 
Stability  Operations 
Civil  Support 

Non-Traditional  Operations 
Access  &  Access  Denial  Operations 
Land  Control  Operations 
Maritime/Littoral  Control  Operations 
Air  Control  Operations 
Space  Control  Operations 


As  we  completed  this  monograph,  it  appeared  likely  that  the  number  of  JCAs  would  be 
reduced  substantially  and  that  the  new  ones  would  align  better  with  the  Joint  Staff’s  Func¬ 
tional  Capability  Boards  than  has  been  the  case  in  the  past.  If  this  occurs,  it  would  be  a 
useful  development. 
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Because  the  JCAs  are  so  broad,  it  is  often  necessary  to  look  to 
their  components  for  something  meaty.  Secretary  of  Defense  Rumsfeld 
approved  a  list  of  Tier  1  JCAs  and  their  Tier  2  components  in  a  memo¬ 
randum  on  May  6,  2005. 

The  decomposition  used  for  the  JCAs  is  both  arguable  and  com¬ 
plicated,  since  there  is  a  great  deal  of  overlap  among  areas,  and  compo¬ 
nents  are  relevant  to  multiple  areas.  Figure  A.l  illustrates  this  for  the 
Global  Strike  problem.*  The  salient  points  are  as  follows: 

•  Global  Strike  is  categorized  as  a  Tier  2  component  of  Strategic 
Deterrence. 

•  Global  Strike  has  a  Tier  3  nuclear  component,  a  conventional 
kinetic  component,  a  non-kinetic  component,  and  a  logistics 
component. 

•  The  conventional  kinetic  component  is  seen  as  “belonging  to”  a 
Tier  2  component  of  a  different  JCA,  the  Strategic  Attack  com- 

Figure  A.l 

Global  Strike  in  the  JCA  Analysis  Framework 
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The  taxonomy  continues  to  evolve.  This  material  is  based  on  Secretary  Rumsfeld’s  memo¬ 
randum  and  a  briefing  by  LTC  Bryan  Luke,  Chief  of  the  Joint  Experimentation  Branch, 
Joint  Staff  (J-7),  dated  August  2006. 
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ponent  of  Joint  Air  Operations.  Joint  Air  Operations,  in  turn,  has 

a  conventional  kinetic  component,  which  “belongs  to”  Strategic 

Deterrence/Global  Strike. 

Although  this  may  appear  a  bit  confusing  at  first,  it  is  an  inevita¬ 
ble  result  of  efforts  to  develop  taxonomies:  The  real  world  is  not  neatly 
hierarchical;  rather,  it  has  various  interactions  across  “branches”  of  a 
hierarchical  tree.  In  any  case,  the  focus  in  this  monograph  is  on  the 
conventional-kinetic  component  of  Global  Strike. 

Other  DoD  offices  have  found  it  necessary  to  use  different  catego¬ 
ries  and  to  construct  complex  mappings  to  the  JCAs.  OSD  (AT&L), 
for  example,  has  noted  that  because  of  the  very  high  degree  of  over¬ 
lap  among  JCAs,  it  has  been  difficult  to  use  them  effectively  for  work 
in  programming  or  acquisition.  OSD  (AT&L)  has  over  the  past  sev¬ 
eral  years  called  for  Capability  Area  Reviews  (CARs)  on  such  topics  as 
integrated  air  and  missile  defense  (IAMD),  joint  battlefield  manage¬ 
ment  command  and  control  (JBMC2),  and  unmanned  aerial  vehicles 
(UAVs),  among  others. 

One  suggestion  has  been  to  lump  the  JCAs  in  different  categories 
as  shown  in  Table  A.  2. 

In  this  construct,  the  enabling  JCAs  enable  warfighting  in  dif¬ 
ferent  kinds  of  engagement.  The  effects  category  refers  to  operational 
effects  in  different  warfighting  domains  (land,  air,  etc.).  Composite 
JCAs  achieve  an  objective  by  drawing  on  both  enabling  and  effects- 
oriented  JCAs.  Institutional  JCAs  are  those  requiring  institutional  sup¬ 
port  for  battlespace  operations. 

Another  attempt  to  map  the  JCAs  to  what  are  seen  as  “practi¬ 
cal”  categories  has  been  made  by  OSD  (PA&E).  The  key  dimen¬ 
sions  in  PA&E’s  breakdown  are  summarized  in  Table  A. 3.  The  three 
major  categories  listed  in  the  table  can  be  seen  as  the  axes  of  a  three- 
dimensional  cube. 
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Table  A. 2 

A  Suggested  Categorization  of  Joint  Capability  Areas 


Category 

Joint  Capability  Area 

Enabling 

Joint  Battlespace  Awareness 

Joint  Command  &  Control 

Joint  Logistics 

Joint  Net-Centric  Operations 

Effects 

Joint  Air  Operations 

Joint  Land  Operations 

Joint  Maritime/Littoral  Operations 

Joint  Protection 

Joint  Space  Operations 

Joint  Special  Operations 

Institutional 

Force  Generation 

Force  Management 

Joint  Interagency/IGO/NGO  Coordination 

Public  Affairs 

Composite 

Joint  Access  &  Access  Denial  Operations 

Joint  Global  Deterrence 

Joint  Flomeland  Defense 

Joint  Stability  Operations 

Defense  Support  of  Civil  Authorities 

Shaping 

SOURCE:  Adapted  from  Kristen  Baldwin  and  Judith  Dahmann,  "JCA 
Implementation  OUSD  (AT&L)  Update,"  unpublished  briefing,  March 
2006. 
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Table  A. 3 

An  Alternative  Breakdown  by  Category 


Major  Category 

Capability  Areas  Relating  Loosely  and 
Imperfectly  to  JCAs 

Role 

Shaping 

Global  Deterrence 

Land  Operations 

Maritime/Littoral  Operations 

Air  Operations 

Space  Operations 

Homeland  Defense 

Stability  Operations 

Function 

Battlespace  Awareness 

Lethal  Force/Effects  Application 

Net-Centric  Operations 

Command  and  Control 

Logistics 

Protection 

Information  Operations 
Interagency/IFO/NGO  Coordination 

Effects  and  Targets 

Military  Adversaries 

Non-Military  Adversaries 

Emerging  Threats  and  Technologies 

WMD  and  WMD  Capabilities 

General  Populations 

SOURCE:  Adapted  from  Porter,  Bracken,  and  Kneece  (2007).  That 
study,  in  turn,  drew  upon  Office  of  the  Secretary  of  Defense/DPA&E 
(2006). 
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Implications  for  Systems  Engineering  and 
Modeling  and  Simulation 


One  task  in  our  original  research  project  for  OSD  (AT&L)  was  to 
examine  family-of-models  issues,  because,  as  discussed  in  Chapters 
Three,  Five,  and  Six,  a  portfolio-analysis  approach  to  capability  assess¬ 
ment  depends  on  knowledge  at  different  levels  of  detail,  sometimes  in 
very  different  forms.  As  it  happened,  much  of  that  research  became 
part  of  our  efforts  for  OSD  (PA&E)  and  a  National  Academy  panel 
(Davis  and  Henninger,  2007;  National  Research  Council,  2006).  We 
shall  merely  touch  upon  that  research  here.  First,  we  discuss  systems  of 
systems  (SoS),  which  are  increasingly  relevant  to  DoD  programs.  The 
Army’s  Future  Combat  System  (FCS),  for  example,  is  an  SoS.  Alterna¬ 
tive  SoS  could  be  evaluated  in  the  kind  of  portfolio-analysis  framework 
we  describe  in  this  monograph. 


A  New  Paradigm  for  Systems  Engineering? 

OUSD  (AT&L)  has  in  recent  times  reemphasized  the  importance 
of  systems  engineering  to  its  work  (Lamartin,  2004;  Wynne  and 
Schaeffer,  2005;  Krieg,  2007).  Others  have  also  commented,  some¬ 
times  in  reviews  of  DoD’s  or  Services’  approaches  to  modeling,  simu¬ 
lation,  and  analysis,  and  sometimes  in  reviews  of  DoD’s  capability- 
development  process  (National  Research  Council,  2004,  2005). 

A  new  wrinkle  in  systems  engineering  is  the  special  problem  of 
SoS.  Although  no  single  definition  has  been  agreed  upon,  we  consider 
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an  SoS  to  be  a  system  many  components  of  which  are  also  full  sys¬ 
tems  in  their  own  right,  perhaps  developed  independently  for  relatively 
narrow  applications.  The  special  features  of  SoS  tend  to  arise  when 
these  component  systems  relate  to  very  different  subject  areas,  in  which 
case  the  SoS  may  be  referred  to  as  having  heterogeneous  components 
and  interdisciplinary  character. 

Although  it  can  be  argued  that  the  original  principles  of  system 
engineering  carry  over  well  to  SoS  work,  some  of  us  are  skeptical  about 
that  perspective  (even  if  it  is  logically  correct),  because — in  practice — 
designing,  acquiring,  and  operating  an  SoS  is  quite  different  from  doing 
so  for  normal  systems.  A  number  of  conferences  and  workshops  have 
now  been  held  on  the  matter,  with  different  conclusions  being  reached, 
depending  on  those  in  attendance.  Table  B.l  summarizes  what  we  see 
as  distinctions  between  old-style  and  new-style  approaches  in  system 
design.  Older  readers  may  note  reference  to  the  changed  conception 
of  “integration.”  For  many  years,  an  “integrated”  system  was  a  clever 


Table  B.l 

Contrasting  Styles  in  System  Engineering 


Issue 

Old  Style  (Stereotyped) 

New  Style? 

Design 

Top-down  with  detailed  specs; 
hierarchical  and  horizontal 
decomposition,  but  not 
modularity 

Top-down  for  architecture,  but 
with  more  open  competition  of 
potential  components;  modularity 

"Integration" 

Components  that  optimize 
performance  for  known  task 
(e.g.,  1970s  compact  stereos) 

Interfaces  with  open  standards 
to  maximize  flexibility  and 
adaptiveness 

Organization 

Tight  internal  control; 
hierarchical;  teams  of  like- 
minded  engineers 

Networking;  control  of  architecture; 

black-box  testing; 
teams  cutting  across  disciplinary, 
social,  and  organizational 
boundaries 

Origin  of 
interface  specs 

Top-down;  design  requirements 

Top-down,  bottom-up,  and 
sideways,  with  satisficing  choices 

Components 

Mix  of  new  and  on-the-shelf; 
long  component  lifetimes 

Mix  of  new  and  on-the-shelf;  fast 
turnover  of  IT  components 

SOURCE:  Paul  K.  Davis,  CSER  06,  panel  on  Complex  Systems  Engineering,  April  7-8, 
2006,  Los  Angeles,  CA. 
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assembly  of  components  developed  to  work  together  extremely  well — 
i.e.,  optimized  for  that  particular  system.  In  today’s  world,  designers 
are  much  less  concerned  about  perfecting  the  interactions  than  they 
are  about  open  interfaces  allowing  for  great  flexibility  over  time.  Those 
involved  in  system  development  for  network- centric  operations  empha¬ 
size  agility  (akin  to  what  we  mean  by  flexibility  and  adaptiveness)  and 
see  the  promise  of  networking  as  very  dependent  upon  being  able  to 
use  many  component  systems  in  larger  systems  of  systems  and  to  vary 
which  components  are  used  and  how  they  interact,  depending  on  the 
circumstances. 


Modeling  and  Simulation 

The  concepts  discussed  in  this  monograph  have  major  implications 
for  analysis  practices  and  for  the  models  and  simulations  on  which 
they  depend.  In  particular,  if  a  defense  program  and  individual  pro¬ 
grams  are  to  be  assessed  for  their  flexibility,  adaptiveness,  and  robust¬ 
ness  (their  FARness),  then  it  follows  that  analysis  must  be  restructured 
accordingly.  Moreover,  analysis  will  need  to  draw  on  M&S  toolkits 
that  are  suitable  for  this  challenge,  rather  than  just  estimating  effective¬ 
ness  in  particular,  detailed  cases.  To  be  sure,  baseline  cases  will  still  be 
very  useful,  and  even  essential,  for  both  integration  and  comparison. 
ITowever,  whereas  the  vast  majority  of  work  time  by  practitioners  of 
M&S  is  currently  spent  on  standard  cases,  future  efforts  will  need  to 
be  much  broader.  The  work  most  badly  needed  for  DoD’s  capability- 
development  efforts  is  more  like  good  high-level  design  than  working 
through  details.  Such  design,  of  course,  may  be  erroneous  unless  it 
is  rooted  in  deeper  knowledge  and  is  supplemented  by  more-detailed 
work,  where  such  work  is  useful.  Thus,  what  is  needed  to  support  analy¬ 
sis  is  a  family-of-tools  approach  to  M&S. 

These  issues  have  been  discussed  elsewhere,  both  by  the  senior 
author  (Davis,  2003;  Davis  and  ITenninger,  2007)  and  by  various 
national  panels  such  as  the  Defense  Science  Board  (Welch,  1996)  and 
the  National  Academy  of  Sciences  (National  Research  Council,  1997, 
2004,  2006).  DoD’s  forthcoming  master  plan  for  modeling  and  simu- 
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lation  will  probably  embrace  the  need  for  a  family-of-tools  approach 
(Allen,  2006). 

Relationships  to  Complex  Adaptive  Systems 

Another  recent  development  has  been  the  bottom-up  emergence, 
within  the  systems  engineering  community,  of  what  is  being  called 
Complex  Systems  Engineering  because  of  the  need  to  draw  upon  the 
paradigms  of  complex-adaptive-systems  (CAS)  theory  to  better  address 
the  modern  issues  of  systems  of  systems.  At  a  recent  conference  on 
the  topic  sponsored  by  Aerospace,  MITRE,  and  RAND,67  one  of  the 
authors  (Davis)  summarized  his  views  on  what  is  special  about  systems 
of  systems  and  how  they  relate  to  CAS. 

The  relevance  of  all  this  to  the  current  monograph  is  that  our 
approach  presupposes  an  underlying  basis  for  evaluation  of  port¬ 
folio  options,  a  basis  rooted  in  good  system  engineering  and  model¬ 
ing.  The  quality  of  that  base,  however,  will  increasingly  depend  upon 
the  analytical  communities  embracing  the  lessons  from  CAS  research 
and  reflecting  the  insights  in  the  variables  and  processes  contained 
within  the  models  and  analyses,  the  components  that  are  recognized, 
and  the  criteria  by  which  evaluations  are  made.  In  this  monograph,  we 
have  emphasized  the  need  to  evaluate  options  for  whether  they  con¬ 
tribute  to  FARness  of  capability.  This  is  precisely  what  is  emphasized 
in  the  first  column  of  Table  B.2.  It  is  very  different  from  what  would 
be  emphasized  in  a  traditional  acquisition  contract  with  industry,  one 
with  highly  detailed  specifications  and  an  emphasis  on  performance 
within  the  narrow  specifications  rather  than  on  FARness. 
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Table  B.2 

Systems  of  Systems  and  Complex  Adaptive  Systems 


Systems  Engineering  and  SoS 


Relationships  to  CAS  Theory 


Although  most  systems  engineering 
principles  carry  over  to  SoS,  many  SoS 
have  unusual  features: 

•  Some  SoS  are  "capability  kits"  allowing 
quick,  at-time,  unique  configuring  of 
situation-specific  systems 

•  Networking  for  tomorrow's  operations 

•  Premium  on  flexibility,  adaptiveness, 
and  robustness 

•  The  right  building-block  components 

•  Superb  assembly  capability 

•  Graceful  degradation 

•  Special  SoS  are  ideal  for  true 
evolutionary  acquisition 

•  Valuable  capabilities  at  each  spiral 

•  Rethinking  of  requirements  at  each  new 
spiral 

•  Certainty  of  capability  expansion,  not 
mere  refinement,  over  time 

•  Certainty  of  need  to  exploit  newly 
proven  technology  components 

Result:  different  conception  of  how  to 
assess  programs,  with  unusual 
categories  and  metrics  and  unusual 
testing 


CAS  theory  is  key  to  distinguishing  SoS 
from  traditional  complicated  systems. 
Characteristics  include: 

•  Hierarchical  vertical  and  horizontal 
components;  networks 

•  People  and  organizations  that  learn, 
innovate,  behave  semi-randomly,  and 
adapt  in  competitive  environments 

•  Diverse,  changing  contexts,  which 
promote  change 

•  "Emergence"  of  patterns  and  operations 
not  explicitly  designed-in 

•  Patterns  of  cooperation  (or  non¬ 
cooperation)  not  originally  intended 

Military  examples: 

•  SOF  officers  working  with  bomber  pilots 
and  horseback  tribal  chieftains  for 
tactical  precision  strikes  on  enemy  forces 

•  Air-mobile/air-assault  Army  forces 
launched  and  supported  from  aircraft 
carriers  after  de-loading  of  some  usual 
Navy  aircraft 

•  Future  ad  hoc  creation  of  tailored 
maneuver  units  for  ground  operations; 
excellent  real-time  networking  with 
local  police  and  military,  with  SOF  and 
intelligence;  exploitation  of  fresh-from- 
laboratory  sensors  and  weapons 


SOURCE:  Paul  K.  Davis,  CSER  06,  panel  on  Complex  Systems  Engineering,  April  7-8, 
2006,  Los  Angeles,  CA. 
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RAND#s  Portfolio-Analysis  Tools 


In  the  mid-  to  late  1980s,  RAND  developed  a  portfolio-analysis  tool 
called  DynaRank  (Hillestad  and  Davis,  1998),  which  has  been  used 
in  a  number  of  projects  for  both  defense  and  social-policy  applica¬ 
tions.  It  continues  to  be  improved  by  colleague  Richard  Hillestad.  The 
Portfolio-Analysis  Tool  (PAT)  was  developed  as  an  offshoot  with  some 
significantly  different  characteristics.  The  first  version  was  the  Portfo¬ 
lio-Analysis  Tool  for  Missile  Defense  (PAT-MD),  which  was  developed 
in  2004-2005  for  the  specific  needs  of  the  Missile  Defense  Agency 
(Dreyer  and  Davis,  2005).  Partly  as  product  improvement  for  MDA 
and  partly  for  the  more  generic  purposes  of  the  research  reported 
here,  Dreyer  and  Davis  have  extended  the  latter  significantly,  creat¬ 
ing  PAT.  Since  only  the  older  PAT-MD  has  published  documentation 
(cited  above),  we  note  the  changes  in  this  appendix,  after  some  general 
comments. 


Applicability  of  PAT 

PAT,  as  illustrated  in  Chapter  Five  in  particular,  is  intended  to 
assist  in  top-down  portfolio  analysis  and  support  of  decisionmak¬ 
ing.  Many  types  of  portfolio  exist,  and  PAT  is  designed  to  be  rather 
generic.  The  common  feature  in  all  applications  of  PAT  is  the  ability 
to  characterize  the  relative  goodness  and  shortcomings,  cost,  and  cost- 
effectiveness  of  alternative  investments  intended  to  contribute  value  in 
a  number  of  different  categories,  such  as  geographic  theaters,  warfare 
domains,  capability  areas,  and  strategic  categories  such  as  warfighting, 
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environment-shaping,  and  laying  the  basis  for  future  large-scale  adap¬ 
tations.  The  contributions  may  be  ambiguous  because  of  uncertainty, 
so  it  may  be  necessary  to  address  different  cases,  such  as  warbghting 
scenarios.  PAT  is  merely  a  spreadsheet  tool  that  assists  in  laying  out 
information  accordingly.  Options  appear  in  rows,  various  measures  of 
goodness  appear  in  columns,  and  the  categories  in  the  portfolio  are 
represented  by  groups  of  columns. 

For  capability-area  analysis,  the  options  that  might  be  compared 
in  PAT  could  range  from  singular  systems,  such  as  a  new  missile  for 
Global  Strike  capability,  to  alternative  approaches  to  systems  of  systems, 
e.g.,  alternative  approaches  to  complex  command-and-control  systems 
extending  across  Services,  nations,  and  even  non-national  groups.  PAT 
is  an  “empty  vessel,”  so  each  application  requires  thinking  through 
the  problem  area  and  developing  appropriate  structures  for  use  in  the 
analysis.  We  have  used  PAT  so  far  in  projects  for  the  Missile  Defense 
Agency,  OSD  (AT&L),  and  the  Joint  Staff.  Earlier  related  work  used 
DynaRank  in  a  study  for  OSD  (Policy)  on  high-level  alternative  strate¬ 
gies  of  a  sort  to  be  considered  in  a  Quadrennial  Defense  Review. 


Major  Improvements  in  Functionality 

The  principal  improvements  in  functionality  relative  to  PAT-MD  have 
been  the  addition  of  (1)  multiresolution  capabilities,  (2)  a  second  level 
of  zoom  (drill-down),  (3)  alternative  formats  for  scorecards,  including 
some  suitable  for  users  who  are  color-blind,  and  (4)  improved  mecha¬ 
nisms  for  adding  or  editing  “perspectives.” 

Multiresolution  Data  Entry 

Using  PAT  requires  inputting  a  great  deal  of  data,  whether  generated 
empirically,  from  models,  or  subjectively.  PAT  also  includes  several 
levels  of  detail,  so  that  the  user  can  observe  results  in  the  summary 
scorecard  and  then  zoom  down  into  detail  for  explanations  of  why  the 
summary  values  are  assessed  to  be  as  shown.  Although  it  may  seem 
ideal  to  have  higher-level  assessments  determined  by  detailed  calcu¬ 
lations,  the  reality  is  that  there  are  often  major  uncertainties  about 
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the  data  for  the  detailed  calculations,  and  even  the  calculations  them¬ 
selves.  In  other  cases,  the  detail  is  spurious  because  it  reflects  specula¬ 
tion  about  underlying  factors  but  not  a  complete  treatment.  Thus,  the 
detail  may  be  useful  conceptually,  but  not  reliable  numerically.  Indeed, 
detailed  data  may  often  be  much  less  reliable  than  more  aggregate-level 
information.  This  is  particularly  so  when  the  aggregate-level  informa¬ 
tion  is  pessimistic.  It  may  be  that  the  detailed  calculations  correspond 
to  a  particular  assumed  mechanism  of  effectiveness,  whereas  the  more- 
aggregate  input  (often  a  judgment)  would  be  based  on  knowing  from 
experience  that  many  detailed  factors  are  at  work,  are  difficult  to  con¬ 
trol,  and  often  cause  trouble.  So,  for  example,  crude  experience-based 
estimates  of  how  long  developments  will  take  may  be  more  reliable 
than  the  result  of  detailed  and  honestly  intended  proposal  estimates. 
This  suggests  the  need  to  be  able  to  input  higher-level  data  directly,  if 
desired,  rather  than  always  proceeding  bottom-up. 

Another  compelling  reason  for  preferring  such  flexibility  in  data 
entry  is  pragmatic:  Inputting  large  amounts  of  data  is  tedious,  and  the 
potential  for  error  is  high.  It  is  often  more  expeditious,  especially  in 
early  parts  of  an  analysis  or  in  response  to  rapidly  changing  assump¬ 
tions,  to  take  the  shortcut  of  entering  higher-level  data  directly. 

Finally,  it  should  be  noted  that  when  experts  review  assumptions 
in  tools  such  as  PAT,  they  look  at  relatively  aggregated  assumptions 
and  compare  them  with  their  experience  and  intuition.  If  they  urge 
using  different  assumptions,  it  is  far  easier — and  may  be  much  more 
sound — to  use  those  directly  rather  than  developing  some  particular 
way  of  disaggregating  them  into  the  terms  of  detailed  calculations. 

To  be  less  abstract,  suppose  that  we  were  evaluating  the  probabil¬ 
ity  of  mission  success  for  deep  strikes,  with  non-stealthy  F-16  aircraft 
being  used  to  penetrate  advanced  air  defenses.  Is  there  any  value  in 
entering  all  the  detailed  performance  data  for  SAMs  and  radars,  the 
observability  characteristics  of  the  F-16s,  and  so  on?  Or  would  it  be 
more  sensible  to  simply  input  low  probabilities  directly,  so  as  to  spend 
precious  time  on  other  parts  of  the  analysis,  such  as  estimating  the  need 
for  various  degrees  of  stealth  in  current  and  next-generation  aircraft?  If 
a  typical  number  calculated  from  detail  were  0.6,  and  an  expert  review 
panel  insisted  on  using  an  even  lower  number,  say  0.3,  one  could  per- 
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haps  do  no  better  than  just  inputting  the  0.3  directly — unless  there 
were  a  sound  basis  in  detail  for  disputing  the  experts’  judgments. 

Another  argument  with  the  same  conclusion  is  that  missions 
change,  the  world  changes,  assumptions  change,  and  it  becomes  very 
difficult,  if  not  impossible,  to  keep  up  if  one’s  decision-support  tool 
requires  excessively  detailed  data.  Further,  even  if  changes  can  be  made 
quickly,  checking  them  out  carefully  may  take  time. 

To  use  another  example,  an  experienced  program  manager  assess¬ 
ing  odds  of  success  in  several  parallel  developments  will  be  drawing 
upon  judgments  about  the  particular  companies  involved,  their  man¬ 
agement  teams,  and  their  recent  records  of  success  or  failure.  His  net 
judgment  may  be  either  more  optimistic  or  more  pessimistic  than  that 
coming  from  a  straight  engineering-level  assessment.  If  the  senior  man¬ 
ager’s  judgment  is  regarded  as  the  best  available  input,  then  it  may  be 
best  to  let  it  be  a  high-level  input  rather  than  to  pretend  that  it  is  the 
result  of  a  more-detailed  calculation. 

PAT  now  allows  users  to  input  options’  effectiveness  data  at  levels 
1,  2,  or  3  of  detail.  Costs  may  also  be  introduced  in  diverse  ways.  For 
example,  one  might  enter  the  annualized  20-year  steady-state  cost  of 
alternative  systems  directly,  or  one  might  instead  have  separate  inputs 
for  R&D,  acquisition  and  deployment,  and  operations  and  mainte¬ 
nance.  The  latter  might  be  totals  for  the  20 -year  period  or  specified  on 
a  year-by-year  basis,  as  in  any  detailed  budget. 

Alternative  Formats  for  Scorecards 

The  default  PAT  approach  is  the  ubiquitous  and  intuitive  color  score- 
card,  with  red,  orange,  yellow,  light  green,  and  pure  green  denoting 
assessments  of  very  poor  to  very  good.  A  significant  minority  of  people 
in  any  given  audience,  however,  may  be  partially  color-blind,  rendering 
the  “intuitive”  displays  useless.  Further,  hardcopy  prints  are  sometimes 
in  grayscale.  Finally,  some  organizations  prefer  other  color  schemes  for 
one  reason  or  another.  PAT  now  allows  the  user  to  choose  from  among 
four  formats.  The  recommended  format  is  the  usual  color  scheme,  but 
with  the  letter  corresponding  to  the  color  also  appearing  in  the  cell  (as 
in  Figure  5.11). 
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Alternative  Perspectives 

A  core  concept  in  using  PAT  is  acknowledging  uncertainties  and 
searching  for  strategies  that  are  flexible,  adaptive,  and  robust.  One  way 
to  go  about  such  exploratory  analysis  is  to  have  alternative  perspec¬ 
tives,  each  of  which  is  a  collection  of  key  assumptions  in  PAT’s  cal¬ 
culations.  The  word  “perspective”  is  apt,  because  frequently,  resource- 
allocation  decisions  are  complicated  by  differences  of  opinion  among 
principals,  differences  that  trace  to  their  differing  perspectives  on  the 
world,  the  future,  and  priorities.68  If  the  alternative  perspectives  are 
defined  appropriately,  having  the  relative  goodness  of  options  hold  up 
across  the  perspectives  is  often  a  measure  of  FARness. 

It  is  now  rather  easy  to  construct  alternative  perspectives  in  PAT 
and  to  compare  portfolio-level  assessments  made  with  different  per¬ 
spectives.  Much  more  progress  can  be  made  in  this  area,  but  the  cur¬ 
rent  version  of  PAT  is  already  quite  useful. 

Comparison  Scorecards 

One  problem  with  color-scorecard  displays  is  that  they  show  results  for 
one  set  of  assumptions,  whereas  comparison  across  assumption  sets  is 
usually  desired.  PAT  permits  side-by-side  comparison  of  scorecards  for 
different  assumption  sets.  The  mechanism  is  simply  defining  a  dupli¬ 
cate  set  of  effectiveness  columns  but  specifying  that  the  values  in  those 
columns  are  those  for  the  alternative  assumption  set. 

Alternative  Aggregation  Formulas 

We  discovered  in  the  course  of  the  Global  Strike  application  that 
PAT’s  alternative  mechanisms  for  aggregating  upward  were  insuf¬ 
ficient.  Although  working  around  this  problem  required  no  changes 
in  PAT  itself,  the  technique  for  doing  so  had  not  been  recognized  or 
documented  previously.  The  method  is  simply  as  follows:  If  the  inten¬ 
tion  is  to  aggregate  from  level  3  to  level  2  using  a  different  combining 
rule  than  is  allowed  by  PAT,  the  user  can  simply  add  an  additional 
level  3  component  called,  e.g.,  net  effect.  The  user  can  enter  the  rel¬ 
evant  formula  in  the  cells  of  that  column,  using  standard  Excel  func¬ 
tionality.  The  formula  can  refer  to  the  component  column  values.  If 
the  component  columns  involve  variables  X,  Y,  and  Z,  the  formula 
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might,  for  example,  be  X*Y*Z  or  1  —  MIN(X,Y,Z).  The  next  step  is  to 
set  the  values  of  the  weights  for  the  contributing  columns  to  something 
very  small  but  nonzero,  such  as  0.01,  with  only  the  net-effect  column 
having  a  weight  of  1 .  The  result  at  level  2  will  then  be  as  specified  by 
the  formula,  but  drilling  down  from  level  2  to  “understand”  the  result 
will  show  the  component  values.  This  technique  is  a  workaround  for 
a  problem  that  only  some  users  would  encounter,  but  it  is  useful  to 
know.  We  do  not  anticipate  changing  PAT  in  the  near  term  to  make 
the  workaround  unnecessary. 

Additional  Columns 

PAT  now  allows  users  to  have  more  top-level  measures  of  options’ 
strengths  and  weaknesses. 

Notes  Sheet 

PAT  now  includes  a  Notes  sheet  in  which  users  are  encouraged  to 
record  assumptions,  musings,  and  hints.  Although  trivial  from  a  tech¬ 
nical  perspective,  this  mechanism  can  be  quite  useful  in  collaborative 
work. 


Simplified  Data  Entry 

PAT  now  includes  a  much  improved  version  of  the  “template-builder” 
tool  described  in  the  original  documentation.  It  is  now  possible  to 
make  substantial  changes,  such  as  adding  or  subtracting  measures  of 
effectiveness,  and  to  generate  the  new  scorecards  while  retaining  all  of 
the  previous  data  that  are  still  applicable.  It  is  necessary  only  to  enter 
new  data. 


Interface  Improvements 

Improving  interfaces  is  a  continuing  effort  over  time.  Hands-on  expe¬ 
rience  with  tools  reveals  interface  problems  that  were  not  appreciated 
originally,  and  as  more  people  use  a  given  tool,  more  ideas  arise  about 
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ways  to  make  working  with  the  tool  easier,  more  transparent,  and  less 
error-prone.  The  improvements  reflected  in  PAT  include 

•  More  intuitive  naming  of  PAT  objects,  such  as  levels 

•  A  top-menu  option  for  navigating  to  other  sheets  (an  alternative 
to  using  the  bottom-of-sheet  tab  mechanism) 

•  Better  defaults  for  fonts  and  font  sizes  in  sheets  and  graphics 

•  Improved  methods  for  making  changes  while  preserving  some  or 
all  previous  data. 


Performance  Improvements 

The  natural  history  of  software  tools  often  includes  a  lengthy  period  of 
early  development  and  experience,  followed  by  an  intensive  period  of 
reprogramming  to  maximize  efficiency.  PAT  is  still  in  the  developmen¬ 
tal  phase,  but  it  has  benefited  from  several  rounds  of  partial  reprogram¬ 
ming,  which  have  substantially  improved  performance.  Calculations 
are  now  much  faster  (e.g.,  seconds  instead  of  tens  of  seconds),  refresh¬ 
ing  graphics  or  changing  displays  or  formats  is  much  faster,  and  the 
effects  of  many  calculations  ripple  through  the  program  automatically 
without  requiring  the  user  to  do  anything.  In  addition,  a  few  bugs  have 
been  identified  and  fixed. 
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1 .  Methods  for  characterizing  and  parameterizing  a  broad  space  are  described  in 
earlier  work  (Davis,  1994,  2002). 

2.  The  rigorous  sufficiency  of  an  intended  spanning  set  depends  on  many  assump¬ 
tions,  e.g.,  that  resources  will  be  used  well  in  any  situation  that  arises,  that  the  low- 
resolution  depiction  has  not  omitted  any  key  dimension,  and  that  the  low-resolution 
depiction  of  a  given  dimension  is  a  conservative  representation  of  more-detailed 
matters.  For  example,  a  test  scenario  characterized  by  a  week’s  warning  time  would 
not  represent  a  real-world  scenario  in  which  the  warning  exists  but  is  not  dissemi¬ 
nated  to  field  units. 

3.  See,  e.g.,  National  Research  Council  (2006)  and  Davis  and  Henninger  (2007). 

4.  Secretary  of  Defense  James  Schlesinger  expressed  this  point  more  than  three 
decades  ago.  It  has  been  raised  more  recently  by  the  Vice  Chairman  of  the  Joint 
Chiefs  of  Staff,  Admiral  Edmund  Giambastiani,  and  the  USD  (AT&L),  Kenneth 
Krieg. 

5.  As  an  example  of  conservatism,  in  the  late  1990s,  DoD  conducted  a  strategic- 
mobility  requirements  study  that  did  not  consider  operational  concepts  and  revised 
military  units  akin  to  what  were  “in  the  air”  and  soon  emerged  as  part  of  military 
transformation.  As  an  example  of  naive  optimism,  the  Army’s  initial  approach  to 
the  Future  Combat  System  (FCS)  depended  on  technologically  unrealistic  estimates 
of  light- armored-vehicle  weights  and  operationally  unrealistic  assumptions  about 
what  could  be  accomplished  with  modern  information  technology  that  was  more 
suitable  for  warfare  in  the  open  than  for  warfare  in  forests  or  urban  settings. 

6.  This  material  draws  on  Chapter  4  and  Appendix  G  of  a  lengthier  Defense  Sci¬ 
ence  Board  discussion  for  which  one  of  the  authors  of  this  monograph  (Davis)  was 
responsible  (Defense  Science  Board,  2007b).  We  benefited  from  discussions  with 
current  and  past  senior  officials,  including  General  Tarry  Welch  (USAF,  retired), 
John  Foster,  Major  General  Jasper  Welch  (USAF,  retired),  Tarry  Lynn,  Admiral 
Dennis  Blair  (US  Navy,  retired),  and  Lt.  General  Glenn  Kent  (USAF,  retired). 
Some  of  Kent’s  insights  are  also  reflected  in  a  RAND  monograph  on  what  might  be 
called  leader-centric  strategic  planning  (Kent  and  Ochmanek,  2003). 
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7.  Chaired  by  General  Maxwell  Taylor,  President  of  the  Institute  for  Defense  Analy¬ 
ses,  the  study  drew  on  top  scientists,  engineers,  and  analysts  from  across  the  defense 
industry.  It  anticipated  greatly  improved  Soviet  strategic  systems  and  evaluated 
alternatives  to  counter  and  trump  those  systems  within  the  paradigm  of  assured 
retaliation. 

8.  One  of  the  authors  (Shaver)  commissioned  RAND’s  first  GPS  study  in  1968. 

9.  The  DDR&Es  of  the  earlier  era  included  Herbert  York,  Harold  Brown,  John 
Foster,  and  William  Perry,  all  of  whom  had  deep  scientific  credentials  and  authority 
just  below  that  of  the  Deputy  Secretary. 

10.  Paul  K.  Davis,  James  Bonomo,  Irv  Blickstein,  and  Robert  Moore,  “Toward  a 
Systematic  Approach  to  Capability  Area  Reviews  (CARs),”  unpublished  material 
prepared  for  Acting  USD  (AT&L)  Michael  Wynne  (December  2004)  and  USD 
(AT&L)  Kenneth  Krieg  (January  2005). 

11.  Developing  metrics  tied  to  operational  missions  is  discussed  in  a  study  on  imple¬ 
menting  the  2001  Quadrennial  Defense  Review  (Kelley,  Davis,  Bennett,  Harris, 
Hundley,  Larson,  Mesic,  and  Miller,  2003). 

12.  A  cynic  might  claim  that  the  result  was  simply  the  consequence  of  organiza¬ 
tional  competition  and  log-rolling.  The  decisions,  however,  were  top-down  national 
choices  that  did  not  neatly  correspond  to  organizational  preferences.  The  Air  Force 
preferred  manned  aircraft,  and  the  Navy  saw  the  SSBN  program  as  a  national  burden 
that  it  was  willing  to  accept  but  not  something  to  be  sought  enthusiastically. 

13.  For  an  early  discussion  using  case  histories  to  draw  lessons  about  why  a  joint 
approach  is  needed,  see  Winnefeld  and  Johnson  (1991). 

14.  We  exclude  reference  to  the  1990-1991  war  with  Iraq  because  that  scenario  had 
been  considered  since  the  late  1970s  and  was  a  significant  factor  in  defense  planning 
that  led  to  the  creation  of  U.S.  Central  Command  and  its  capabilities  (Department 
of  Defense,  1991). 

15.  See  Kugler  (1993)  for  an  excellent  history  of  the  NATO  alliance. 

16.  See  Thomson  and  Gantz  (1987),  an  influential  paper  of  its  time. 

17.  These  RAND  analyses  were  done  for  OSD  (Policy)  (Ben-Horin,  Darilek,  Jas, 
Lawrence,  and  Platt,  1986;  Davis,  1998;  Darilek  and  Setear,  1990).  Instead  of  fixat¬ 
ing  on  standard  scenarios,  the  1998  study  argued  that  a  short-mobilization  attack 
was  the  largest  military  risk  borne  by  NATO,  that  Soviet  planners  might  reasonably 
conclude  that  their  best  chance  for  rapid  victory  would  be  with  such  an  attack — 
coupled  with  political  deception — to  achieve  a  degree  of  operational  surprise,  and 
that  the  feasibility  of  such  an  attack  could  be  undercut  by  verifiable  constraints  on 
the  readiness  of  Soviet  operational  reserves. 
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18.  The  reasons  for  Milosevic’s  eventual  capitulation  may  never  be  known.  Some 
historians  make  a  case  that  the  cumulative  and  increasingly  pointed  effects  of  strate¬ 
gic  bombing  were  decisive,  with  other  factors  contributing  (Hosmer,  2001).  Others 
are  more  equivocal  (Daalder  and  O’Hanlon,  2000;  Lambeth,  2001),  discussing  how 
strategic  bombing,  diplomacy  such  as  Russian  recommendations  to  Milosevic,  and 
the  threat  of  an  eventual  ground-force  invasion  interacted  and  jointly  resulted  in 
the  decision. 

19.  See  Davis  (2002)  for  discussion  of  capabilities-based  planning  and  scenario- 
space  methods  for  going  about  it. 

20.  See  Bexfield  (2004)  on  the  Analytic  Agenda.  See  Swett  (2003)  for  a  discussion  of 
how  OSD  (Policy)  approached  issues  of  uncertainty  in  scenario  planning. 

21.  The  new  generation  of  precision  weapons  not  only  further  improved  the  target- 
kill  potential  of  bombers  using  older  precision  weapons,  but  also  allowed  them  to 
strike  multiple  targets  per  sortie,  thereby  going  far  beyond  what  had  been  possible 
previously. 

22.  Enthusiasts  of  long-range  fires  and  airpower  may  argue  that  the  Army  can  forgo 
heavy  artillery,  but  ground-force  officers  know  of  historical  cases  where  air  forces, 
which  might  have  protected  them  in  principle,  were  temporarily  not  available.  Also, 
there  can  be  a  critical  need  for  significant  levels  of  indirect  fire — in  part,  to  cope 
with  enemy  infantry  approaching  through  complex  terrain  (Matsumura,  Steeb, 
Herbert,  Lees,  Eisenhard,  and  Stitch,  1997;  Matsumura,  Steeb,  Gordon,  Herbert, 
Glenn,  and  Steinberg,  2000;  Defense  Science  Board,  1998). 

23.  See,  e.g.,  Alberts  and  Hayes  (2003)  for  a  discussion  of  mission-capability 
packages. 

24.  The  latter  problem  is  sometimes  egregious  in  methods  eliciting  subjective  judg¬ 
ments  about  the  adequacy  of  a  system’s  parts  from  various  people  within  the  orga¬ 
nization  and  then  using  standard  decision-analysis  software  to  tote  up  the  results 
using  linear-weighted  sums.  Not  surprisingly,  these  judgments  are  often  looked 
upon  unfavorably  by  top  officers  or  officials,  who  want  to  know  who  made  them  and 
the  logic  behind  them. 

25.  See  Kelley  et  al.  (2003)  for  a  discussion  of  finding  metrics  to  correspond  to  guid¬ 
ance  in  the  Quadrennial  Defense  Review. 

26.  In  terms  of  2007  dollars,  the  Soviets  had  spent  on  the  order  of  $600  billion  on 
their  air-defense  system  as  of  1989  (Lepingwell,  1989). 

27  Some  of  the  best  early  research  on  this  matter  was  accomplished  for  the  OSD’s 
Office  of  Net  Assessment  in  the  mid-1980s  (Hines,  Petersen,  and  Trulock,  1986). 
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28.  A  significant  literature  exists  on  these  matters  (Quinlivan,  1995;  Dobbins, 
McGinn,  Crane,  Jones,  Lai,  Rathmell,  Swanger,  and  Timilsina,  2003).  The  Army 
War  College  and  other  organizations  also  have  extensive  related  databases. 

29.  See  Davis,  Kulick,  and  Egner  (2005)  for  a  survey  of  modern  decision  science  and 
its  implications  for  high-level  decision  support.  Davis  and  Kahan  (2007)  discuss 
applications  to  effects-based  operations. 

30.  After  a  RAND  review  of  its  processes,  the  Missile  Defense  Agency  (MDA) 
established  a  mechanism  encouraging  such  communication  across  the  boundary  of 
investment-planning  and  systems-engineering  domains.  This  proved  valuable  during 
2005  and  2006  in  assuring  that  top-level  assessment  and  development  of  programs 
was  consistent  with  the  best  technical  knowledge  from  systems  engineering. 

31 .  This  point  can  be  appreciated  from  a  discussion  in  an  AI AA  volume  on  modeling 
and  simulation  for  space  systems  prepared  by  the  Aerospace  Corporation  (Rainey, 
2006). 

32.  For  a  broad  discussion  of  classic  models,  see  Hughes  (1989).  EADSIM’s  web  site 
states  that  EADSIM  is  the  most  heavily  used  force-on-force  model  in  existence.  See 
http://www.eadsim.com/EADSIMExecSum.pdf. 

33.  See  Allen  (2006)  for  community  input  coordinated  by  the  Institute  for  Defense 
Analyses  describing  the  several  campaign  models.  For  CAMMD,  see  Willis, 
Bonomo,  Davis,  and  Hillestad  (2006).  For  EXHALT,  see  McEver,  Davis,  and 
Bigelow  (2000).  For  EXHALT-CF,  a  closed-form  version  of  EXHALT,  see  Davis, 
McEver,  and  Wilson  (2002). 

34.  See  National  Research  Council  (2006),  Davis  and  Henninger  (2007),  and  Davis 
and  Bigelow  (1998). 

35.  The  efficient-frontier  method  in  financial  analysis  is  a  special  application;  it 
was  pioneered  in  the  1950s  (Markowitz,  1952)  and  was  subsequently  improved  by 
Sharpe  and  others  (Sharpe,  2006).  In  financial  applications,  the  efficient  frontier 
usually  refers  to  a  curve  on  a  plot  of  expected  profit  from  a  stock  portfolio  versus 
some  measure  of  risk,  such  as  the  variance  to  be  expected  in  the  value  of  the  port¬ 
folio  over  time.  A  better  measure  of  risk  looks  only  at  the  probability  of  negative 
fluctuations  from  the  expectation. 

36.  DynaRank  does  not  generate  a  simple  “priority  list”  of  the  sort  often  used. 
Instead,  it  evaluates  options  across  multiple  criteria  and  uses  potential  contributions 
to  net  effectiveness  and  cost  to  suggest  the  order  of  investments  to  meet  the  con¬ 
straints  of  a  budget.  Tie  result  can  be  seen  as  a  kind  of  priority  list  but  is  informed 
by  cost-effectiveness  considerations. 

37  The  Joint  Staff  has  also  used  a  simplified  version  of  Global  Strike  to  describe 
aspects  of  analytic  methodology  at  an  unclassified  level  (Joint  Staff  (J-8),  Force 
Application  Assessment  Division,  2006). 
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38.  See  Rumsfeld  (2006);  Cartwright  (2006);  International  Institute  for  Strategic 
Studies  (2006);  Arkin  (2001). 

39.  A  combatant  commander  might  recommend  a  particular  global  strike  or  might 
be  asked  to  plan  and  conduct  one,  but  deciding  on  the  strike  itself  would  be  a  presi¬ 
dential  issue. 

40.  For  discussion,  see  unpublished  work  by  MITRE  and  Joint  Staff  (J-8)  (Yost, 
2006).  See  also  a  Joint  Stalf  doctrine  study  (Joint  Chiefs  of  Staff,  2005,  p.  III. 3), 
which  says  that  global  strikes  are  rapidly  planned,  limited-duration,  extended-range 
precision  attacks  that  are  conducted  to  achieve  strategic  objectives.  Global  strikes 
may  be  executed  against  highly  valued  adversary  assets,  using  both  kinetic  and  non- 
kinetic  methods.  Global-strike  targets  include  adversary  centers  of  gravity,  WMD 
and  their  delivery  systems,  production  facilities  and  storage  sites,  key  leadership, 
and  critical  infrastructure.  Other  examples  of  homeland-defense  offensive  actions 
include  (special  operations)  direct  action,  space-negation  denial,  and  computer- 
network  attacks. 

41.  For  broad  conceptions  of  Global  Strike,  see  early  comments  by  the  Air  Force 
Chief  of  Staff  (Jumper,  2001)  and  a  Joint  Staff  integrating  concept  (Joint  Staff 
(J-8),  2004). 

42.  Some  related  discussion  can  be  found  in  a  recent  paper  speculating  about  the 
feasibility  of  Israel  attacking  Iranian  nuclear  facilities  (Raas  and  Long,  2006).  Such 
an  attack  could  release  radioactive  materials.  Worries  about  release  of  toxic  chemi¬ 
cals  after  bombing  arose  in  U.S.  deliberations  before  the  Desert  Storm  operation 
in  1991,  and  some  public  literature  exists  about  toxic-release  issues  as  a  result  of 
Gulf  War  Syndrome  investigations.  See,  e.g.,  CIA  Director  Robert  Walpole’s  testi¬ 
mony  in  2000  (on  DoD’s  web  site  at  http://www.gulflink.osd.mil/oversight/xcript_ 
hearing_  1 3  j  ul99.  html#cia) . 

43.  The  August  20,  1998,  strike  against  Afghan  camps  illustrated  the  potential  value 
of  promptness  in  Global  Strike.  It  was  hoped  that  the  attacks  would  kill  bin  Laden, 
who  was  expected  to  be  present  at  one  camp  (Clarke,  2004,  p.  184).  Although  bin 
Laden  was  not  killed,  some  reports  indicated  that  he  had  left  only  shortly  before 
the  strike;  if  this  was  in  fact  the  case,  a  shorter  execution  time  might  have  made  a 
difference.  The  facts  remain  murky,  and  according  to  President  Clinton’s  account, 
“we  never  knew  for  sure”  (Clinton,  2004,  p.  803).  The  strike  had  been  planned  for 
days  as  the  result  of  intelligence  that  bin  Laden  would  be  present  in  the  camp,  and 
execution  time  may  have  mattered.  Other  efforts  to  hit  bin  Laden  were  stood  down 
for  reasons  of  uncertainty  and  likely  collateral  damage,  but  if  conducted,  they  might 
well  have  been  execution-time-limited.  This  can  be  inferred  from  accounts  of  the 
period  (Benjamin  and  Simon,  2002,  pp.  280ff). 

44.  The  1998  strike  into  Sudan  was  based  on  what  many  saw  as  marginal  data  and 
inference;  it  was  severely  criticized  both  within  the  administration  and  by  outsiders. 
See  “To  Bomb  Sudan  Plant,  or  Not:  A  Year  Later,  Debates  Rankle,”  New  York  Times 
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on  the  Web,  October  27,  1999  (http://query.nytimes.com/gst/fullpage. html?res=9A0 
4E7D71238F934A15753C1 A96F958260).  As  of 2004,  President  Clinton  stated  that 
he  “still  believed  we  did  the  right  thing  there”  (Clinton,  2004,  p.  805).  This  judgment 
reflected  an  after-action  review  that  he  requested  when  he  himself  became  doubtful. 
An  account  by  NSC  staffers  of  the  original  evidence,  criticism,  and  subsequent  review 
makes  clear  how  difficult  choosing  strike  targets  can  be  and  how  difficult  it  can  be  to 
persuade  critics  that  attacks  were  made  in  good  faith,  let  alone  with  correct  judgment 
(Benjamin  and  Simon,  2002,  pp.  352-362). 

45.  About  20  people  were  killed  in  Damadola,  according  to  news  reports  from  Paki¬ 
stan.  Of  those,  some  may  have  been  members  of  al  Qaeda.  The  strike  was  accom¬ 
plished  with  a  Predator  drone,  according  to  a  Congressional  Research  Service  report 
to  Congress  (Katzman,  2006). 

46.  As  an  example,  President  Clinton  sought  opportunities  to  strike  against  bin 
Taden  after  the  1998  embassy  bombing  in  Kenya.  When  opportunities  arose, 
lengthy  evaluation  processes  were  needed,  because  of  risks  such  as  the  potential 
for  one  strike  to  kill  members  of  the  royal  family  of  the  United  Arab  Emirates.  See 
testimony  of  CIA  director  George  Tenet  to  the  9/11  Commission  (National  Com¬ 
mission  on  Terrorist  Attacks,  2004). 

47.  At  one  point  in  the  bin  Laden  hunt,  the  Navy  was  directed  to  hover  off  the  coast 
of  the  Arabian  Sea  in  hopes  that  an  opportunity  to  strike  would  arise.  Pakistani 
patrol  boats  motored  out  to  the  fleet  and  reported,  with  the  information  presumably 
being  relayed  to  the  Taliban.  Subsequent  “hovering”  relied  on  submarines  (Benja¬ 
min  and  Simon,  2002,  p.  280). 

48.  An  analogy  can  be  drawn  to  planning  for  nuclear  strikes  by  manned  bombers 
during  the  Cold  War.  Electronic  countermeasures  (ECM)  can  be  extremely  effective 
in  defeating  air  defenses.  Through  the  1970s,  however,  it  was  customary  for  related 
studies  to  be  done  separately  and  for  capability  assessments  related  to  bomber  pen¬ 
etration  to  be  conducted  without  assuming  such  countermeasures.  The  reasoning 
was  simple:  ECM  are  often  fragile  capabilities,  dependent  on  good  intelligence  and 
on  the  adversary  not  knowing  their  capability;  ECM’s  effectiveness  can  sometimes 
be  undercut  quickly  (on  the  timescale  of  development)  and  possibly  without  the 
side  possessing  the  ECM  knowing  about  it.  The  B-1B  was  eventually  procured  with 
more  explicit  dependence  on  ECM  than  earlier  planners  had  recommended,  but  the 
wisdom  of  that  course  of  action  is  still  debatable. 

49.  In  2007,  the  Senate  Appropriations  Committee  expressed  concerns  about  the 
CTM  and  other  prompt  Global  Strike  options  and  asked  for  a  National  Academy 
Study  on  the  subject.  The  committee  members  expressed  particular  worry  about 
what  they  saw  as  ambiguities,  i.e.,  whether  launches  from  a  Trident  submarine 
would  be  perceived  as  nuclear.  They  also  expressed  skepticism  about  the  availability 
of  adequate  intelligence  to  justify  prompt  attack. 


Endnotes  151 


50.  Some  logically  possible  cases  are  excluded  as  unnecessary.  For  example,  a  small 
point  target  might  be  buried  but  not  hard. 

51.  For  a  related  discussion,  see  recent  NRC  reports  (National  Research  Council, 
2005,  2006)  and  a  white  paper  developed  for  DoD’s  master  plan  for  modeling  and 
simulation  (Davis  and  Henninger,  2007). 

52.  Multiresolution  modeling  has  been  discussed  at  some  length  elsewhere  (Davis 
and  Bigelow,  1998). 

53.  During  the  Clinton  administration,  serious  efforts  were  made  to  strike  bin 
Laden.  The  combination  of  shortcomings  in  information  and  high  degrees  of  risk 
caused  the  president,  in  each  case,  to  forgo  the  attack.  Critics  who  claim  that  the 
information  justifying  a  strike  was  adequate  might  ponder  the  miserable  record  of 
intelligence  during  Operation  Iraqi  Freedom,  when  numerous  strikes  were  made, 
starting  with  the  dramatic  effort  to  kill  Saddam  Hussein  himself.  Repeatedly,  the 
intelligence  information — including  that  from  human  sources — proved  faulty,  and 
the  strikes  failed  to  accomplish  their  goal.  Moreover,  they  caused  numerous  unin¬ 
tended  fatalities. 

54.  The  intention  is  to  minimize  the  time  spent,  while  conveying  the  essence  of 
the  subject.  Multiple  viewgraphs  of  assumptions  take  time  to  present  and  explain 
and  therefore  have  a  high  opportunity  cost  and  the  potential  for  “turning  off” 
recipients. 

55.  As  an  example,  early  in  the  campaign  in  Afghanistan,  air  strikes  were  perceived 
as  being  too  indiscriminate  to  justify  their  modest  accomplishments,  and  the  United 
States  was  being  criticized  harshly,  despite  the  perceived  legitimacy  of  the  attack 
into  Afghanistan  following  September  11.  The  criticism  dissolved  as  airpower  began 
supporting  effective  ground  operations  by  allied  tribesmen  working  with  U.S.  SOF 
units  or  intelligence  operatives.  It  became  clear  that  world  opinion  would  “tolerate” 
collateral  damage  if  it  were  seen  as  unavoidable  in  accomplishing  something  legiti¬ 
mate.  Empirical  information  on  such  matters  is  discussed  in  a  recent  RAND  study 
prepared  for  the  U.S.  Air  Force  (Larson  and  Savych,  2006). 

56.  Despite  the  oft-heard  claim  that  the  Services  never  offer  up  tradeoffs,  the  Navy’s 
conventional  Trident  Missile  Concept  (TCM)  would  reduce  marginally  the  number 
of  strategic  nuclear  ballistic  missiles  on  alert  in  deployed  Trident  boats.  That  is,  some 
TCMs  would  replace  nuclear  missiles,  rather  than  requiring  an  additional  subma¬ 
rine  or  new  launch  tubes.  A  partial  bill-payer  would  be  on-alert  nuclear  capability. 
The  Air  Force,  in  proposing  a  new  long-range  bomber  (or  a  medium-range  bomber, 
or  both)  suggests,  e.g.,  a  substantial  reduction  in  the  number  of  legacy  bombers 
(B-52s  and  B-lBs).  Congress  is  reluctant  to  retire  the  older  aircraft  (Tirpak,  2006). 

57  RAND  analyses  for  the  U.S.  Air  Force  by  colleagues  Bill  Stanley,  Carl  Rhodes, 
and  Richard  Mesic  have  generated  plots  of  this  general  character,  but  the  plots 
shown  here  are  purely  notional. 
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58.  An  example  from  another  domain  is  relevant:  We  have  seen  the  abject  failure 
of  extremely  expensive  private-sector  and  government  efforts  to  develop  high- 
functioning,  reliable  information-technology  systems.  This  has  been  a  problem  for 
the  Internal  Revenue  Service,  the  FBI,  and  major  commercial  banks,  among  other 
oganizations. 

59.  A  number  of  the  possible  programs  for  Global  Strike  five  to  15  years  from  now 
have  considerable  technical  risks.  The  Air  Force  and  the  Army,  for  example,  are 
both  considering  programs  to  build  sophisticated  missiles  with  boost-glide  capabil¬ 
ity.  Such  missiles  are  quite  plausible,  but  some  of  the  enabling  technologies  are  at  an 
early  stage  of  development. 

60.  The  controversial  history  of  strategic  bombing  is  analyzed  in  a  book  by  Robert 
Pape  (Pape,  1996).  The  most  successful  example  of  strategic  bombing  is  probably 
that  in  the  conflict  over  Kosovo,  although  assessments  differ  on  the  relative  effects 
of  the  bombing,  the  threat  of  ground-force  invasion,  and  intervention  by  Russia 
(Flosmer,  2001;  Tambeth,  2001). 

61.  Some  of  the  difficulties  are  described  in  recent  work  done  as  part  of  a  Ph.D. 
dissertation  by  Zoltan  Jobbagy  (Jobbagy,  2006).  That  reference  also  contains  an 
extensive  bibliography.  For  a  discussion  of  analytical  issues,  see  also  Davis  (2001) 
and  Davis  and  Kahan  (2007).  For  an  official  description  of  effects-based  operations, 
which  is  notable  in  part  for  its  failure  to  address  uncertainty,  see  a  recent  monograph 
by  US  JFCOM  (Joint  Warfighting  Center,  2006). 

62.  Some  care  must  be  taken  in  estimating  these  risks,  because  even  if  the  new  capa¬ 
bility  provided  by  an  option  fails  in  practice,  the  ability  to  accomplish  the  mission 
may  not  be  too  severely  affected  when  alternative  concepts  of  operations  are  avail¬ 
able  that  would  not  need  those  capabilities. 

63.  MDA  requested  that  RAND  review  its  approach  to  investment  planning,  relat¬ 
ing  it  to  capabilities-based  planning,  as  discussed  in  earlier  RAND  work  (Davis, 
2002).  An  unpublished  report  on  that  review  led  to  a  project  on  investment¬ 
planning  methods  and  tools. 

64.  The  mission  statement  specifies  a  layered  defense  and  capability  for  defense  in  all 
phases  of  flight.  Such  specificity  belongs  in  a  solution  rather  than  a  mission  statement, 
but  it  reflects  the  drafters’  desire  to  require  flexibility  and  depth  of  capability. 

65.  Such  issues  are  discussed  in  a  recent  academic  study  (American  Physical  Society 
Study  Group,  2003). 

66.  This  reflects  discussions  with  Tt.  General  Ronald  Kadish  (USAF,  retired)  and  Tt. 
General  Trey  Obering  (USAF),  the  current  and  past  commanders  of  MDA. 

67  The  Symposium  on  Complex  System  Engineering  was  organized  by  Russ  Abbott 
of  Aerospace  and  California  State  University,  Eos  Angeles  and  supported  by  Aero¬ 
space,  MITRE,  and  RAND.  It  was  held  at  RAND’s  Santa  Monica,  Calif.,  head- 
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quarters  in  January  2007.  A  very  brief  discussion  of  conference  themes  is  given  in 
a  briefing  by  Abbott  posted  on  his  web  site,  http://cs.calstatela.edu/wiki/images/b/ 
bl/Abbott.ppt. 

68.  Multiresolution  data  entry  is  closely  related  to  multiresolution  modeling 
(Bigelow  and  Davis,  2003;  Davis  and  Henninger,  2007;  National  Research  Coun¬ 
cil,  2006). 
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